Re: non recursive makefiles

[MJ Ray]

Nicola Pero,

Thank you for continuing this discussion. Thanks to the others too.
I note the points about -j, namespaces and subprojects made and will reply
to them in context of Nicola's message.

Dealing with the last point first: I was not accusing the current
maintainer of not maintaining it enough. It seemed a popular opinion
that the current design isn't documented as much as it could be and I've
struggled to link the Documentation with what exists at the minute,
so I concluded that there are people who would like to help who don't
understand the current code. There's more love which it could get.

What is the Master idea? Why can't we set the environment and then exec
the user's preferred shell again instead of sourcing GNUstep.sh? Those
are probably obvious questions and I didn't find answers. I know shell
and some make, but I don't know gnustep-make and it's complex enough
to deter me.  I admit I am rusty on this. My first published computing
article was about applying graph theory to compilation, but I've not
had opportunity to work on this for years.

Nicola Pero <nicola@brainstorm.co.uk> wrote:
> Well, I tried to explain in words.  You can't iterate in make.  You can't
> execute pieces of code iteratively -- you can only execute them once, or
> never.  this does not make it exactly easy to iterate over files, read
> each of them, and then execute the same rules for each of them with
> different variables.  You CAN NOT execute the same rules with different
> variables in the same make invocation, which is what we want to do here.

Are you sure it's what we want to do? It's what we would want to do
if we were using an imperative programming language like Objective-C
to drive the build, but we agree that make is not a full programming
language.

According to its manual, GNU make "constructs a dependency graph of all
the targets and their prerequisites." To that graph, the above situation
needs to look like the final stage of building the files *depends
on* having completed the previous stage for all files. Why can't that
dependency be declared, rather than relying on implicit execution order
in the version of make used? I think this may be why using -j is breaking
for some people. Have I understood Helge's point correctly?

> Moreover, there are no local variables or rules.  Everything is global.  

Yes, this point is understood. There will be times when we need to
do something specific with variables or have two subprojects that
interfere with each other. Is this the general case, though? If not,
can we accommodate the special cases without penalising the general?

In the few GNUmakefiles I've just looked at, it seems that things
could be rewritten with prefixes or +=, or are already using prefixes.
Some things, like APP_NAME, can already take a list. Most packages don't
do anything more than the minimum of setting a few things and doing some
includes - is that generally true?

> Btw, how much better ... are you sure that that is what gives you slow
> build times ? [...]

Even ignoring the point about -j, having to revisit every directory on
a large project that was already built is a performance problem too. One
feature of make is that it can do just the work needed to rebuild.

> > Yes, the subprojects that you use in one project have to be compatible,
> > but that's already true.
> No, it's not.  You can do
> ADDITIONAL_OBJCFLAGS += -lPincoPallino
> in one GNUmakefile, and that will *not* be seen by another GNUmakefile in 
> another directory.

If you are messing with the general *CFLAGS for something which should
not be passed to all final outputs, isn't that asking for trouble?
Why not AppName_OBJCFLAGS += -lPincoPallino (or similar) if it's only
for that app?

> > Recursion doesn't change that fundamentally either.
> Of course it does.  ADDITIONAL_OBJCFLAGS defined in subproject A have no
> effect on ADDITIONAL_OBJCFLAGS defined in subproject B because they are
> read in different, sandboxed, make invocations.

Depending on their relative positions in the hierarchy, subproject A can
pollute subproject B even across make invocations. It's not global, but
allowing it's still a bug, isn't it?

Some variables are reset with some make invocations, but it still
alarms me to see it called "isolation" or "sandboxing".

> > Not isolation at all. Assuming isolation seems dangerous to me.
> I don't agree.  "Isolation" is a basic design pattern of good software
> architecture, and it looks like an essential requirement for a maintable,
> usable system to assume isolation of subproject makefile code.

You are assuming isolation where it doesn't exist, which is the problem.
How can a subproject's make be "isolated" from all other makefiles
yet still allow settings passed from the environment?

> > Recursion does allow "dirty" makefile practices to continue longer. Maybe
> > sometimes recursion would be necessary, but I don't see why it seems to
> > be the default for so many GNUstep packages. Is it just lack of time to
> > hack an elegant solution, which is what your message suggests?
> 
> No - it's the way that make works that forces you to do this.

Are you sure? The gnustep-make DESIGN file looks ideal for using a
straight make rather than recursing.

> make is supposed to let you define rules and targets and use them.  (btw,
> generally, make does encourage you to use recursive invocations to iterate
> over directories).

Make encourages you and then penalises you for doing it? I'd take
the advice of such an awkward friend with caution ;-)

Why do you think it's encouraged? It's documented, but that might be to
stop people doing it badly. The case given as an example in the manual
is fairly narrow. Other people are very scathing about it, such as
"Recursive make Considered Harmful" by Peter Miller (1998), Journal of
AUUG, http://aegis.sourceforge.net/auug97.pdf

I think that gnustep-make is better than automake, but I wonder if more
speed can be obtained by looking again for simplicity and seeing if
any of the special case complexity can be refactored.  More efficient
rebuilds and more use of -j seems desirable. The DESIGN looks sound,
but I can't relate it to the code very well.

I acknowledge the edge case of incompatible projects, but I still
haven't seen anything which looks like a general problem.  The problem
causing iteration over files looked solvable by other means, but maybe
I misunderstood it.

I await any replies with interest.

-- 
MJR/slef