Re: hidden dependencies

[Krzysztof Cieniuch]

Warlich, Christof wrote:
> Hi,
>
> I had to write a new build system using GNU make only for a rather complex and 
> big project which had a
> really werid and unmaintainable build system nailed up from a bunch of perl 
> scripts and a few makefiles.
>
> As everything was almost finished now, I just gave GNU make's -j option a try. 
> Doing so, I had to learn that I
> obviously missed a dependency here and there, not causing issue so far as make 
> generated everything in
> a certain sequence that made everything availabe in time more or less by 
> coincidence of doing things in the
> right sequence.
>
> While it is no problem to fix my build system to include these missing 
> dependencies, I wonder that there may
> still be more missing dependencies "under the hood", which may show up any 
> time, e.g. whenever timing
> changes during a parallel build of when rules are reordered.
>
> My question: Is there a generic best practice approach to find all existing 
> dependencies that are hidden
> by a more or less arbitrary build sequence?
>
> Thanks for any ideas,
>
> Chris
> _______________________________________________
> Help-make mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/help-make
>   
If you are using recursive make then the answer is no it is not possible 
even on "make level"
google for document "Recursive Make Considered Harmful" your parallel 
builds will randomly fail
and the risk of failing will increase as you increase number of jobs.

But even if you do not use recursive make then getting automagically all 
build internal dependency information is pretty much out of scope
of make e.g. simple scenario: generated sources.
If somewhere in the build you generate code and in other parts you 
include generated headers make obviously won't
detect source code dependencies that only C/C++ preprocessor knows of.
If during build you automatically generate dependency info as described 
on Paul Smith pages
after full build you could then grep all dependency info and see which 
modules depend on generated headers
and then add that dependency  explicitly but any change in source code 
(i.e. adding in any source file  #include  for generated file) would require
updating manually dependency information.
Each set of tools can potentially  have different type of internal build 
dependencies that are hard to predict until build failure.

I'm very interested in this subject.  In my project we have generated 
code for rpc interfaces and corba plus there is perl tool that generates 
some xml files.
In Makefiles this dependency must be explicitly specified by developer 
there is special variable that defines  generated sources deps
e.g. t1_GSRCDEPS:=generated_srcs_id   (there is advanced resources 
definition mechanism in place so in makefiles you do not use source file 
names just
some magic ids :-) defined elsewhere)
I didn't figure out how to automatically at build time detect and issue 
an error if some module is missing dependency information.

Thanks
Krzysztof