Re: build system rules & algorithms

[Mike Shal]

On 6/13/09, grischka <address@hidden> wrote:
>  Here is one from a test with TUP's "make_v_tup" case, 10000 files,
>  for the "nothing changed" case, times in seconds:
>
>  make                   11.42
>  make -r                 3.83
>  read makefile           1.76 (913431 bytes = 0.5MB/s)
>  git status              0.27
>  tup upd                 0.01
>
>  You see that "git status" is very fast to teil the changes, of course
>  not as fast as TUP which knows the changes in advance.  But I'd assume
>  that this is roughly the time needed to "stat" the files (on linux).
>
>  Obviously GNU make spends most of the time with trying built-in
>  pattern rules (you can see it with "make -d ...").  Of course the
>  test case does not require that feature (as most other makefiles
>  out there don't either).

Doh - thanks for pointing that out. I normally disable the built-in
stuff in my own Makefiles, but clearly neglected to do that in those
tests. I'll have to update that on the web page to give make a fair
fight :)

>
>  If we disable built-in rules then the big part is reading the makefile.
>  (To measure I did just stick a "$(error eof)" at the end of it).
>  0.5MB/s is a solid value with room for optimization. (TCC reads and
>  compiles 20-30MB/s).
>
>  For the rest of 1.8 seconds (3.84 - 1.76 - 0.27) we can assume that
>  a part of it is for "processing the DAG in the alpha way".
>
>  However we just don't know how big that part is.  As opposed to "read
>  makefile" and "git status" it does not call out for external resources,
>  so even if it's O(10000), it might be very fast actually, maybe under
>  0.1 secs.

Good idea there with the $(error eof). If I can't get a profiler to
get better statistics, maybe I'll just try to stick a few targeted
gettimeofday() in the source to try to pinpoint the DAG processing
time.

>  Well, have to say that with the monitor and auto-dependency features
>  TUP gives a really magic impression at first look.  Just the input
>  language seems rather limited atm.  What about abusing GNU make as
>  a preprocessor for makefiles->tupfiles?  In experience the tools that
>  succeed are the ones with working from/to conversions.

I'm trying to keep the input language simple, so it's easy for new
folks to come into a project and be able to figure out how files are
created (as in, go to a directory where a file is created, and open
the Tupfile there). Though if you think there are specific
deficiencies that makes it hard to write them, I'd like to hear about
it! (Of course that's not particularly suitable for the make mailing
list).

So far when trying to build other projects with tup, I just run make
in verbose mode and paste the commands into the Tupfiles along with
their inputs and outputs. After it builds, I refactor the Tupfiles to
try to make them more readable. A lot of that can probably be
automated, but I'm not sure a full-fledged converter is entirely
feasible.

Thanks!
-Mike