Re: Can we do it: variable scoping

[Boris Kolpackov]

Hi Paul,

Paul D. Smith <address@hidden> writes:

> And, you can even clear out variables after you've included a makefile,
> using the $(.VARIABLES) special variable, to avoid leaking variables
> between included makefiles for example.

I have a fairly big, non-recursive, multi-makefile build system and
I do exactly that. I have a special include function that saves
variable values (in other, ugly-named, variables) before the actual
inclusion and restores them afterwards. It is slow and very hard
to debug.

So, I agree, variable scoping would be great. At a minimum, support
for makefile-local, in addition to global, scopes would make life
much easier. Something along these lines:

# base.mk
push

foo := bar
foo:
        @echo $(foo)
pop

# makefile
foo := foo

include base.mk

all: foo
        @echo $(foo)

This would print:
bar
foo

A variable that is referenced in the local scope is first searched
for in the local scope and then in the global scope.

Once you have that, the next natural thing to want is to use values
from one scope in another. For example, the included makefile may
need to have some local variables but may also want to set some
global ones to be used by the including makefile (and use the 
local values for it). This one is a bit tricky syntax-wise since 
we need to somehow specify that a variable is from the outer scope.
I like the colon as a separator but this may interfere with the 
target syntax. Maybe we can use double-colon instead (target-
specific varaible assignment using double-colon appears to be
legal but maybe we can re-purpose it):

# base.mk
push

foo := bar
tmp := $(::foo)   # Use global foo
::baz := $(foo)   # Set global foo

pop

Once we have this, what would be great is to have named scopes ;-).
This will basically be a syntactic sugar for the common pattern
where a global variable is prefixed with a "subsystem" name. So,
for example, instead of writing this:

%: %.m4
        @$(m4_cmd) $(m4_flags) -o $@ $<

m4_cmd := m4
m4_flags := -q

One could write:

push m4

%: %.m4
        @$(cmd) $(flags) -o $@ $<
pop

m4::cmd := m4
m4::flags := -q

-or-

push m4
cmd := m4
flags := -q
pop

Maybe we could also use curly braces instead of push/pop:

m4 {

cmd := m4
flags := -q

}

> I think, actually, that it's not that hard for the most part.  We
> already have the ability to have nested variable scopes, to support
> target-specific variables.  This feature would create multiple "global"
> scopes as well, and individual targets would be hooked to the scope that
> was active when the target was defined.  So a new scope would create a
> new variable set as the "global" scope, and all targets would be hooked
> to that as their variable scope.  When the scope ended we'd "pop" that
> scope and reinstall its parent as the global.  This is all already
> basically supported internally.

That sounds good. It seems we will also get the proper variable lookup
semantics out of this (i.e., if a variable is not defined in the inner
scope, the next outer scope is checked, and so on).


> I thought we might also need "variable assignment scopes", as well as
> target scopes, but I'm not sure we do: I can't think of any situation
> where a variable is evaluated in a deferred context where it would need
> to know what scope it was defined in, other than in a target context.  I
> know that didn't make much sense but I can explain further if anyone
> cares.

Yes, please. I am not sure what "variable assignment scopes" means.


> There are three issues I can see: first, we need to come up with a
> makefile syntax.  I think a simple (nesting) push/pop-style scope is
> what we want (I definitely don't want to get into "named scopes").

That's too bad. I think without them the functionality will be quite
limited. For example, there would be no way to use a variable from 
one scope in a definition of another variable in another scope.



> Does anyone know of any other make implementation (or similar build
> tool) that implements something like this?

Last time I looked at build tools, I haven't seen anything like this.


> Second, we have to think carefully about how a new variable scope would
> interact with the existing "target specific variable inheritance"
> feature.  This could be the thing that knocks down the entire proposal.
> Basically we already have a stack of variable scopes, constructed
> dynamically based on the prerequisite tree.  How does a static variable
> scoping, tied to the target itself, fit into that existing stack?
> 
> For example, today if we have:
> 
>       foo: FOO = foo
>       foo: bar
> 
>       FOO = bar
>       bar: ; @echo $(FOO)
> 
> you'll get "foo" here, because the "bar" target inherited the value of
> FOO from foo's target-specific variable.  If you run "make bar" you'll
> get "bar".
> 
> How does this kind of thing work if you add variable scoping?  Obviously
> expanding "FOO" first looks at bar's target-specific variables, but if
> it's not there where does it look next?  Does it look at the parent
> target's variable list, which might be in an entirely different variable
> scope?

I agree this is somewhat of a misfeature but I think the consistent behavior
would be to hook the variable scopes at the end of the target's list. I.e.,
first look in all parent targets and then in the variable scopes of the
child target. The good thing is we now can mark a variable private to 
disable the inheritance.

Boris

-- 
Boris Kolpackov, Code Synthesis        http://codesynthesis.com/~boris/blog
Compiler-based ORM system for C++      http://codesynthesis.com/products/odb
Open-source XML data binding for C++   http://codesynthesis.com/products/xsd
XML data binding for embedded systems  http://codesynthesis.com/products/xsde