Re: Optimization for reading *.d files

[Michael Stahl]

On 19.03.2017 09:54, Norbert Thiebaud wrote:
> On Sat, Mar 18, 2017 at 9:25 PM, brenorg <address@hidden> wrote:
>>
>> Anyway, I want to continue to use GNU Make, and not fallback to CMake/Ninja.
>> After some profiling, what's killing me is parsing the "*.d" files generated
>> by the compiler.
>>
>> The time to include all dependency files of my project in one single
>> makefile (as I want to get rid of recursive make), is 4 seconds.
>>
> 
> have you looked in how much redundancy you have in all these dep ?
> For LibreOffice, which use one big make to build it all  (60k files or so)
> we wrote a step to combine and de-duplicate all these .d file
> reducing the amount that need to be parsed by make by quite a bit
> 
> iow instead of getting a faster process, reduce the amount to be processed.

that can still be improved: Bjoern implemented something quite similar
to what Breno wants, an "includedepcache" keyword and
optimized/restricted dep file format, as previously discussed on this list:

https://mid.mail-archive.com/address@hidden/msg08784.html

but it's probably not used by anybody currently since we don't want to
require a feature that's not in upstream GNU make.

the implementation is somewhere in here:

https://gerrit.libreoffice.org/gitweb?p=gnu-make-lo.git;a=shortlog;h=refs/heads/gnu-make-lo-4.0