Re: Variable definition in eval'd function

[Bryan Ischo]

[Sorry if this is a re-send, I wasn't subscribed to address@hidden the
last time I tried to send it, and I am not sure if my previous version is
going to get through]

address@hidden wrote:
> Bryan Ischo-3 wrote:
>> --- begin foo.mk ---
>> VARIABLE = foo_variable
>> --- end foo.mk ---
>>
>> --- begin bar.mk
>> VARIABLE = bar_variable
>> --- end bar.mk ---
>>
>> --- begin Makefile ---
>>
>> FRAGMENTS = foo bar
>>
>> define TEST_RULE
>>
>> .PHONY: $(1)
>> $(1):
>>        @echo $$@
>>        @echo $(2)
>>
>> endef
>>
>> define PROCESS_FRAGMENT
>>
>> -include $(1).mk
>>
>> $$(eval $$(call TEST_RULE,$(1),$$(VARIABLE)))
>>
>> endef
>>
>> $(foreach i, $(FRAGMENTS), $(eval $(call PROCESS_FRAGMENT,$(i))))
>>
>> --- end Makefile ---
>>
>> This allows me to use a single variable, VARIABLE, that is defined in
>> foo.mk and bar.mk, and each makefile fragment results in the rules that I
>> was hoping for, i.e.:
>>
>> .PHONY: foo
>> foo:
>>         @echo foo
>>         @echo foo_variable
>>
>> .PHONY: bar
>> bar:
>>         @echo bar
>>         @echo bar_variable
>>
>
> Can you please explain why is it necessary to $(eval) twice - once in
> PROCESS_FRAGMENT and once in $(foreach)? Thanks.

Mr. Hoolam (or is it Mr. Woon?) -

I'm sending my response to your question to the help-make mailing list
because my answer (and my further comments and suggestions on how to
improve GNU Make to make all of this much simpler) might be interesting to
a wider audience.

I still get confused about the specifics of GNU make syntax, because it
gets very complicated when $(eval) and $(call) are used.  But I'll do my
best to get it right here ...

The thing to remember about $(call) is that it's a text function.  When
GNU Make evaluates a $(call) function invocation, the result is just text.
 GNU Make does not necessarily evaluate the text for the purpose of
building rules, assigning variables, or any of the other things that GNU
Make does when evaluating makefile text.  So when using $(call), you most
always want to use $(eval) to instruct GNU Make to evaluate the resulting
text as makefile syntax.  Thus the only place I see $(call) being useful
outside of being used in conjunction with $(eval), is when doing variable
assignments.  Because you either want to treat the text that results from
$(call) as makefile syntax for GNU Make to evaluate, or as text that would
go in a variable.  I am not sure that there is any other way that $(call)
would be useful.  In other words I see these two meaningful useage
patterns with $(call):

SOME_VARIABLE = $(call ...)
$(eval $(call ...))

Correspondingly, the $(eval) function is not a text function; its result
is not text, but instead its result is "nothing", with the *side-effect*
of having GNU Make actually create the rules and variable assignments and
stuff that the evaluated text describe.

Now as to why the double $(eval $(call)) syntax is needed ...

When GNU Make encounters the outer $(foreach) call, it causes GNU Make to,
for each value in the $(FRAGMENTS) variable, do the action in the
rightmost part of the $(foreach).  That action is "$(eval $(call
PROCESS_FRAGMENT,$(i)))".  So it's going to evaluate the result of
applying the text function $(call) to the template PROCESS_FRAGMENT with
the single template argument $(i) (which is the next value from the
$(FRAGMENTS) variable).

$(call) simply replaces all instances of $(1) in the text of
PROCESS_FRAGMENT with the first template argument, which was the foreach's
$(i) variable, which itself was the next value from the $(FRAGMENTS)
variable.

The result is this text, assuming that the template argument's value was
"foo", would be:

-include foo.mk

$(eval $(call TEST_RULE,foo,$(VARIABLE))

Now the outer $(eval) goes to work on this text.  It first includes the
foo.mk file, which results in VARIABLE being assigned the value
"foo_variable".  Then it sees an $(eval) function call, which means that
it is now going to evaluate the result of $(call
TEST_RULE,foo,foo_variable), which means substituting "foo" for $(1) in
the TEST_RULE text, and "foo_variable" for $(2) in the TEST_RULE text.
The resulting text is:

.PHONY: foo
foo:
     @echo $@
     @echo foo_variable

The $(eval) function gets a hold of this and GNU Make processes these
directives into the corresponding rule.

Once that is done, the inner $(eval) is done.  And once that is done, the
outer $(eval) is done also since there's nothing else for it to do.

Now, what if there were no outer $(eval)?  Then the result of:

$(call PROCESS_FRAGMENT,$(i))

would have just resulted in the text:

-include foo.mk

$(eval $(call TEST_RULE,foo,$(VARIABLE))

But GNU Make never would have processed it as makefile syntax since there
was no outer $(eval) to tell it to do that with the result of the $(call).
 This is where I think that GNU Make's syntax is bad and needs to be
improved.  It is nonsensical to do nothing with the resulting text that
$(call) produced, so it doesn't make any sense at all to not have $(eval)
wrapping $(call) here.  I don't know what GNU Make tries to do with the
text that results from the $(call) if it isn't subsequently sent to
$(eval), but I think it just gives some cryptic error.  I think that
$(eval) and $(call) should be combined into a single function, call it
$(calleval) for lack of something better.  Or maybe just call it $(eval),
and let the syntax look something like this:

$(eval TEMPLATE,arg1,arg2,...)

The effect being to replace $(1) with arg1, $(2) with arg2, etc, in the
text  TEMPLATE, and then to evaluate the result.  This would I think give
absolutely all of the functionality that the current syntax gives, but
would be less confusing.  But I digress.

So we can see that the outer $(eval) is necessary to get the results of
the outer $(call) to be evaluated as GNU Make syntax instead of just being
some resulting text that GNU Make doesn't do anything with and likely just
emits an error about.

Now as to the inner $(eval) ... if it wasn't there then the result of the
outer $(call) would look like this:

-include foo.mk

$(call TEST_RULE,foo,$(VARIABLE))

This would be the text that the outer $(eval) ended up evaluating.  And
this means that all the outer $(eval) would do would be to include foo.mk,
resulting in the assignment VARIABLE=foo_variable, and then run the
$(call) function to substitute "foo" and "foo_variable" into TEST_RULE. 
The resulting text would be this:

.PHONY: foo
foo:
     @echo $@
     @echo foo_variable

But GNU Make would not evaluate it as makefile syntax!  Once again this
would just end up as garbage text that GNU Make would probably issue an
error about (or worse - just ignore, which would be even more confusing). 
Once again an $(eval) is needed to get this text evaluated as makefile
syntax.  That is the reason for the inner $(eval).

So to summarize: without the outer $(eval), the result of the $(foreach)
call when handling "foo" would just be this text which GNU Make would not
have been directed to evaluate as Makefile syntax and would either barf on
or just ignore (both are bad, the latter much worse so):

-include foo.mk

$(eval $(call TEST_RULE,foo,$(VARIABLE))

And without the inner $(eval), the result of the $(foreach) call when
handling "foo" would just be this text which GNU Make would not have been
directed to evaluate as Makefile syntax and would either barf on or just
ignore (both are bad, the latter much worse so):

.PHONY: foo
foo:
     @echo $@
     @echo foo_variable

I think that GNU Make could be improved in three ways to make all of this
*much* easier:

1. Replace $(eval $(call)) with a single $(calleval) function which does
both the template text replacement of $(call) and the GNU Make syntax
processing of $(eval) at the same time (and I would go one step further
and just name the function $(eval), completely replacing the functionality
of the existing $(eval) and eliminating the $(call) function entirely from
GNU Make).

2. Have GNU Make automatically handle the result of any $(calleval) as
makefile text to be further parsed and handled by GNU Make, instead of
generating an error or ignoring the text.

3. Change the way that variables are handled in rules; I'm still pretty
confused on this but I do know that the way that GNU Make (it inherited
this behavior from the original make) handles variables in rule commands
is counterintuitive; they don't end up getting replaced until the very
last step, when the rule is actually applied, which means that this
doesn't work like you'd think it would:

VARIABLE=foo_variable
.PHONY: foo
foo:
     @echo $(VARIABLE)
VARIABLE=bar_variable
.PHONY: bar
bar:
     @echo $(VARIABLE)

The above actually results in rules that look like this:

foo:
     @echo bar_variable
bar:
     @echo bar_variable

The reason being that VARIABLE isn't substituted with its current value
every time the makefile syntax defining the rules is encountered; but
instead it is substituted when the rules are being post-processed (at the
time that they are used during the actual execution of the makefile I
believe, not during the parsing phase), and VARIABLE already has its final
value.  I think that this behavior is totally degenerate; one can always
fix the value of a variable to be the same in two different rules by using
a variable that is only assigned once and not changed between rules; but
when one changes a variable's value between rules, I can't think of any
reason to have done that *except* for the new value to be used in
subsequent rules.  I understand that changing this behavior would break
badly-written makefiles, but perhaps a flag could be passed to GNU Make to
cause it to use the new semantics, and hopefully someday, a switch over to
making the new semantics the default.

If these three changes were made to GNU Make, then my example could simply
be (in its entirety):

--- begin foo.mk ---
VARIABLE = foo_variable
--- end foo.mk ---

--- begin bar.mk
VARIABLE = bar_variable
--- end bar.mk ---

--- begin Makefile ---

FRAGMENTS = foo bar

define PROCESS_FRAGMENT

-include $(1).mk

.PHONY: $(1)
$(1):
       @echo $@
       @echo $(VARIABLE)

endef

$(foreach i, $(FRAGMENTS), $(calleval PROCESS_FRAGMENT,$(i)))

--- end Makefile ---

I think that would be much more logical, and that correspondingly more
complex GNU Make files would benefit even more (remember that my example
here was a very simple one!  complex GNU Makefiles using $(eval) and
$(call) extensively can get very hairy indeed!).

Thank you, and best wishes,
Bryan


------------------------------------------------------------------------
Bryan Ischo                address@hidden            2001 Mazda 626 GLX
Hamilton, New Zealand      http://www.ischo.com     RedHat Fedora Core 5
------------------------------------------------------------------------