Re: Any way to get rid of -MP parameter to gcc for dependency creation?

[Ralf Wildenhues]

* Xan Lopez wrote on Tue, Jan 04, 2011 at 02:46:55PM CET:
> On Tue, Jan 4, 2011 at 2:21 PM, Xan Lopez <address@hidden> wrote:
> >
> > Without the local hack to get rid of the -MP flag a null-build with
> > that version of GNU make is ~40s. CVS HEAD gives ~26s (wow!), and CVS
> > HEAD with the "get rid of -MP" hack gives ~14s. So definitely there's
> > been a big improvement in Make in the last year, thanks for pointing
> > that out. It still seems worth to try to improve the -MP case though,
> > 26 > 14 is a big win.
> 
> One last data point. In the best case I've got so far (14s), time tells me:
> 
> make  8.27s user 5.86s system 99% cpu 14.264 total
> 
> If I'm right this means I've moved from having the vast majority of
> the time in user CPU time to a 60/40 split, which I guess means by now
> we are spending a lot of time stating files (?).

Can you send sysperf output for this as well, please?  Thanks.


The makefile + included files that I've looked at (where not all of
webkit is built) already weigh in at some 90 MB total size, the very
bulk of which in the .deps/*.P*o files just as you mentioned.  Getting
that down to 50some MB by not using -MP seems desirable iff make really
is spending most of its time reading, parsing, and not stating.
Of course it also seems desirable to not lose the deleted header hack.

With GNU make, the effect of -MP can actually be emulated with a
catch-all pattern rule like this:
  %.h: ; @:

(assuming that all involved headers match the pattern %.h).

This is not the first issue where it would be nice to special-case GNU
make for better efficiency and better warnings.  I think Automake should
provide a special GNU_MAKE conditional (maybe name it AM_GNU_MAKE or so,
since the other name is already taken by the gnulib module gnu-make),
and allow for some special behavior being triggered by that:

- Code enclosed in 'if GNU_MAKE/endif' sections could be processed with
  GNU make-portability warnings turned off.

- We could even let a project declare that it always requires GNU make
  (that way turning off related warnings completely).

- '-MP' could be avoided and instead catch-all rules like above be used
  if the package developer also takes care to define the set of header
  extensions (using a variable, say AM_HEADER_EXTENSIONS); we could then

     if GNU_MAKE

     am__dashMP =
     am__dummy = -no-dummy

     define _am_deleted_header_fix
     %.$(1): ; @:
     endef
     $(foreach ext,$(AM_HEADER_EXTENSIONS), \
         $(eval $(call _am_deleted_header_fix,$(ext))))

     else
     am__dashMP = -MP
     am__dummy =
     endif

  and of course also set a flag for depcomp to simplify the output for
  non-GCC compilers.

I think I might be able to do a proto patch over the weekend.


For the remaining 14s, there is one more uuugly hack I can think of:
get rid of the recursive make invocation that is done in the 'all' rule.
The reason it exists is so $(BUILT_SOURCES) and 'autotoolsconfig.h' are
updated before the "normal" prerequisites of 'all' are dealt with.
Can you require GNU make 3.80 or newer?  Then I think the following
(completely untested!) addition should cut the time almost in half:

# Ugly hack: override automake's definition of 'all' in order to
# avoid a make recursion.
# This requires order-only prerequisites (GNU make 3.80 or newer).
all: all-am
%.$(OBJEXT): | $(BUILT_SOURCES) autotoolsconfig.h
%.lo: | $(BUILT_SOURCES) autotoolsconfig.h


After that, it's probably time to start writing a beta build system.

BTW, it seems webkit also has a cmake-based build system (right?).
Does what it creates fare better?

Cheers,
Ralf