Re: Suggestion for a change in the out of date algorithm

[Paul Smith]

On Sun, 2007-06-03 at 12:17 +0200, Ramón García wrote:
> Here is a couple of ideas for the out of date evaluation:

Hi Ramon; thanks for keeping us up to date.

One thing: I'm sure you're already considering this but I'd like to
request that the "out of date" mechanism be abstracted from the rest of
the code, as much as possible.  So, ideally, the current mechanism would
be abstracted to one or two function calls that define a comprehensive
API for "out of date detection".  Right now this detection is a bit
scattered.

> Instead of passing the list of targets and sources, pass a pair
> (target, source) and evaluate the rule for each pair.
> 
> The rationale is that the user model is more restrictive, because we
> assume that the out of date is always an and of out-of-date(target,
> source_i) for each source. This restriction makes simpler for the
> user. Consider writing a user defined rule for a pair, and writing it
> for a list of sources. The list handling by hand will make it more
> difficult for the end user to write such rules. I can't imagine a
> single case where this flexibility is useful.

I don't know that the list handling is that much more difficult, but I
do agree that I don't see any situations where treating the
prerequisites list as a whole is required.  It can be more efficient in
some cases, as you mention, but not required due to functionality.  Of
course, we can still allow $^ to be set to the entire list, even if we
evaluate $< for each element of the list, just in case people want it.

> Let us see the issue of cases where the out-of-date determination
> should be different for different prerequisites of the same target
> 
> A.class: A.java B.class C.class
> 
> A.java does not depend on A.java and B.class in the same way. If
> something in the implementation of A.java changes, it should be
> recompiled, but the content of B.class and C.class is only relevant
> for the interfaces that they export, rather than the implementations.
> 
> For handling cases like this, a new extension is proposed to make
> expressions: a function to evaluate an expression in a different
> context. If we call this function change-expression,
> ${change-expression target-name expression} will evaluate expression
> in the context of target-name. Now, to solve cases like the previous
> one, we declare some expression in a different way for different
> prerequisites, and this expression is invoked from the .OUT_OF_DATE
> rule using the change-expression macro.

Hm.  I think the idea is useful, but the details suggested above seem
too specific to me.  In other words, I can imagine there might be a
number of different situations where you might want to be able to
evaluate a variable in the context of a file (a "different scope" as it
were) that don't have anything to do with out of date determination.

Maybe it would be better to use a different syntax for this.  I
understand the appeal of using functions, because they're easy to use
for extending make.  But I wonder if a more "built-in" syntax might be
better here.  Maybe this:

        $(<target>::<variable>)

?  I chose this because ":" is already a special character in variable
references, since you can do things like $(VAR:.c=.o) or whatever...
plus this syntax is common in many languages.  I don't think this syntax
is already valid for anything useful.  In the above case the variable
would be evaluated in the context of the file <target>.

> Now, how can we implemented these "changed" macros?
> 
> For handling persistent state of compilation rules, I suggest a
> behavior close to the one proposed by Paul, but slightly more general.
> I propose a macro
> 
> changed-value <name> expression
> 
> This macro returns true of the value of expression evaluated in the
> previous make invocation is different from the current one. In
> addition to returning true, the value obtained by this invocation is
> stored. <name> is used for making it easier to manage persistent
> stored values, so that different invocations of changed-value in
> different places of the Makefile do not clash because of different
> values of <name>.
> 
> For instance, changed could be implemented as:
> 
> %: changed=${changed-value md5 ${md5sum  address@hidden
> 
> %.class_interface: %.class
> %.class_interface: changed=${changed-value md5_intf  ${shell
> get-java-interface $< | md5sum}}
> 
> We use a hypotetical command get-java-interface that reads a .class
> files and returns a file that contains interface definitions only.

The thing that concerns me about this method is that it seems to me like
you're evaluating the changed variable again every time a file is used
as a prerequisite.  That is, suppose in your example that
B.class_interface was a prerequisite to 10 different .java files; as far
as I can tell you'll be re-expanding the changed rule for B.class 10
times, even though it's obviously sufficient to expand it only once,
since it can't have changed after the first time.  This seems very
wasteful indeed.

There are ways to avoid this obviously, by adding more special functions
or whatever.

But I wonder if things have gotten too complex in the search for
flexibility.  I'm not adverse to limiting flexibility in less-important
ways if it makes the user's interface much simpler.

For example, maybe we can say that for any given file, it will have only
one value that determines its up-to-date-ness.  Then, instead of
evaluating that value again every time the file appears as a
prerequisite, make would automatically evaluate that value once, after
the file was treated as a target and its commands were successfully
invoked.

Then that value could be expanded many times without being recalculated.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <address@hidden>          Find some GNU make tips at:
 http://www.gnu.org                      http://make.paulandlesley.org
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist