[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: hidden dependencies

From: Krzysztof Cieniuch
Subject: Re: hidden dependencies
Date: Fri, 21 Jan 2011 11:53:19 +0000
User-agent: Thunderbird (X11/20101213)

Philip Guenther wrote:
On Thu, Jan 20, 2011 at 2:35 AM, Krzysztof Cieniuch
<address@hidden> wrote:
In that previous project, each module generated either a library (with
matching header(s)) or a program.  The Makefile for a library recorded
the include path for accessing the headers it supplied, while the
Makefile for any module that depended on other modules' libraries had
to declare which modules it depended on, so that it could both get
access to the depended upon modules' include files (via their recorded
include paths)...and so that the Makefile system could declare the
order-only dependencies involving all the involved generated header

My build system includes only necessary makefiles when building project
module (subdirectory)
Parsing each time 500 makefiles is not an option since it takes to long (on
Sun T5440 it takes around 2.5 minute) and if developer is working
on single module edit/recompile cycle would take to long.

(2.5 min?  Wow, T5440's are even suckier than the hardware that
project had to deal with!)

Yeah they are terrible developer build boxes.
For comparison parsing makefiles for the same project on xeon takes 11 seconds. and on Itanium
with roughly the same clock 28 seconds.
But to be fair it is not that bad when building from scratch it takes roughly the same time to compile it has 256 virtual cores it is real pleasure to watch make -j256 in work :-)))
Although it took a while to develop tools that would put back build output
as you would invoke single threaded recursive make build system

If each directory declares the directories that it has a dependency
on, then you could restrict your Makefile inclusions to those that are
necessary for the (active) targets.  It does get a bit tricky to get
right, I'll admit.  Once your Makefile setup gets complex enough it
requires testing and debugging like any other program...

My build system works exactly as you described there is set of core makefiles
the build system "brain" and few hundred project makefiles which are not
really makefiles but more like build directives.
Example makefile in module that builds shared library looks like this:

include mk/init.make
t1_LIBDEPS:=lib1 lib2
include mk/make.make

where LIBDEPS lists not real library names but resources ids.
So if dependencies location in project changes it is just a matter to updating resource definition. Also there is feature to allow "import" selected resources from one project to other.

There is a lot of different features as well :
you may have multiple targets in one Makefile.
you may mix c++/c with idl and rpc definition sources like this: file2.c file3.idl file4.x
you may set compile options per file e.g.
t1_SRCS:=[no_optimize] [obj:file1_c]file1.c
(by default object file names will be derived from source file but in above sample
we would have name clash so file1.c will be compiled into file1_c.o )
There is global INSTALL_FILES directive for easy resources copying
INSTALL_FILES:=file1 [chmod:755]                          $(INSTALL_DIR) \
file2 [chmod:644,mv:file2_copy] $(INSTALL_DIR) \ file3 [ln:file3] $(INSTALL_DIR)
Automatic dependency generation including command arguments
i.e. if command to build target  changes the target is rebuild
and so an and on ...  (enough showing off :-))

Each time developer makes a build only necessary makefiles are included and parsed so make logically 'sees' only one makefile. Since all rules are generated this is quite intensive process. But there is caching and debugging mechanism that allows to print generated rules instead of invoking them i.e. you may direct build system so instead rule evaluating it prints them using $(info ). So it easy to verify if generated rules are correct or troubleshoot why something doesn't work as expected. Note this is different from -n switch since that debug output is fully valid makefile with all the generated recipes. It can be used to build project this is how caching works. The size of cached makefile in project root is about 11M and then even on T5440 it takes 20 seconds to parse that makefile instead of 2.5 minute.

I think it would be worth to put some good ideas or best practices into make manual instead of
chapter "5.7 Recursive Use of make"  :-)


reply via email to

[Prev in Thread] Current Thread [Next in Thread]