[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Partial checkout builds
Re: Partial checkout builds
Fri, 22 Oct 2010 21:25:13 -0400
I would suggest trying UnionFS (supported either natively, or via FUSE
on most *NIXen). What you do is mount /release/src under /local/src,
with /local/src/ on top and /release/src on bottom. UnionFS first
checks if a file exists on at the top layer, and keeps checking the next
layer until it finds the file, or goes through all the layers.
top: /local/src: util.h m1.cc
bottom: /release/src: util.h m1.cc m2.cc
Depending on which kernel you're on, you can either do the mount directly in
or you may have to create a separate directory (as you do in Linux).
on Linux (via FUSE) the command to mount is:
unionfs-fuse -o cow /local/src/=RW:/release/src/=RO /build/src
on FreeBSD the command is:
mount -o union /release/src /local/src
mount_unionfs -o below /release/src /local/src
I hope this helps!
On Fri, 2010-10-22 at 11:54 +0100, Krzysztof Cieniuch wrote:
> What I'm trying to do is to make partial source tree builds working.
> This is useful in release/patches scenario.
> When developing software time to time one needs to make release :-)
> After release development continues on head and released version is
> tagged in source control system and is "frozen".
> Eventually it will be deployed in the field and then will come defect
> reports (yes it has bugs :-)
> so one needs to prepare patches for that particular release version.
> One way to do that is to checkout tagged version fix issue rebuild
> affected module (test) and ship to client.
> So if you need to prepare many patches/enhancements for the released
> product one ends up with multiple checkouts of released version.
> The better solution would be to have released source checkout available
> on network drive (with read only permissions) and be shared by all
> So when you need to patch module A you just checkout that module A (or
> even better just files from that module that need to be modified)
> and setup your build environment so that any missing files in your local
> checkout will be taken from common sources on network drive.
> I got all bits and pieces working thanks to make and vpath the only
> thing remaining is proper behavior of c/c++ preprocessor.
> Here is example, let say we need to fix module A so after analysis it
> turns out that one need to modify two files from that module so
> developer source tree looks like this:
> And the release source tree on network drive:
> Note both m1.cc and m2.cc include util.h like this:
> #include "util.h"
> Now if one modifies util.h make will correctly try to update both m1.o
> and m2.o (and then build library from those objects)
> m1.o will be build from correct files i.e.
> but m2.o will be build form incorrect set of files:
> /release/src/mod1/m2.cc <-- ok! : we do not have m2.cc checkout locally
> /release/src/mod1/util.h <-- problem! : preprocessor will first look in
> current directory of m2.cc and will use release version of util.h
> instead of our patched version
> So it is strictly c/c++ preprocessor problem not make but since most of
> the people on this list are using make to build something :-)
> maybe you have some ideas how to solve this issue.
> I tried to employ -I- option but that works fine only in simple
> scenarios and one must be very strict how include files are included
> (don't want to go into details here)
> so could be implemented if one have full control over sources but then
> come system, thirdparty headers that cannot be modified.
> Using -I- option I ended up with huge list of directories that need to
> be included via -I option and header name clash problems.
> Now I'm in the middle of implementing file io wrapper library that would
> be preloaded when running compiler and would provide
> merged view of release and local directory to preprocessor (some poor
> mans virtual file system) but I need to support three platforms Linux/Sun/Hp
> so it is not trivial and besides is not very portable (and it will be
> very hard to verify correctness of that solution)
> My question is is there way to fix that preprocessor issue or maybe
> different approach that would make partial sources builds possible.
> Help-make mailing list