Re: [Qemu-devel] [PATCH] makefile: detect corrupted elf files

[Michael S. Tsirkin]

On Sun, May 26, 2013 at 07:28:40PM +0000, Blue Swirl wrote:
> On Sun, May 26, 2013 at 6:24 PM, Michael S. Tsirkin <address@hidden> wrote:
> > On Sun, May 26, 2013 at 06:20:17PM +0000, Blue Swirl wrote:
> >> On Sun, May 26, 2013 at 1:40 PM, Michael S. Tsirkin <address@hidden> wrote:
> >> > On Sun, May 26, 2013 at 02:36:28PM +0100, Peter Maydell wrote:
> >> >> On 26 May 2013 13:31, Michael S. Tsirkin <address@hidden> wrote:
> >> >> > On Sun, May 26, 2013 at 10:12:21AM +0100, Peter Maydell wrote:
> >> >> >> I definitely think individual project makefiles are the wrong place
> >> >> >> to fix this. If create-as-temp-and-rename is useful functionality
> >> >> >> it needs to go in the compiler so that everybody benefits.
> >> >> >
> >> >> > This will not help users on existing systems.
> >> >> > Also it's not just compiler. We'd have to do it in linker,
> >> >> > asm, ... lots of work.
> >> >>
> >> >> This is clearly less work than implementing it in the makefile
> >> >> of every single open source project in the world (or even every
> >> >> single open source project in Debian).
> >> >
> >> > You seem to have removed the part that explained that
> >> > 1. we run scripts in our makefiles so need to handle that anyway
> >> > 2. we care about users on existing systems
> >>
> >> A generic hook (default none, or maybe "test -s") after object
> >> production and before linkage should be enough but would scale to SHA
> >> producing/verifying tools.
> >>
> >> >
> >> > This means that we would need the fix in our makefiles even
> >> > if compiler and linker gain this feature.
> >>
> >> Depends on the feature. If the object files have robust checksums
> >> which are checked after output and before input, this should be
> >> transparent to the build system.
> >>
> >> >
> >> >> > You are wellcome to implement this in compiler/linker/etc if you like
> >> >> > but we will still want to handle it in our makefile as well.
> >> >>
> >> >> I specifically don't want it handled in our makefiles because
> >> >> it's the wrong place to fix the problem and it will make
> >> >> our build system more complicated.
> >>
> >> +1
> >>
> >> Also, what is the worst case scenario? The link fails and you have to
> >> clean up and rebuild? An automated build system can't produce the
> >> expected output if the build machine is unreliable?
> >
> > It's a simple issue.
> > Each time I reboot during build, I have to make clean and rebuild.
> > This wastes my time so I looked for ways to save the time.
> 
> Compile under a stable kernel and test the bleeding edge kernel only
> as KVM guest? Get a different box for testing or compiling? Run 'sync'
> every time gcc finishes?

What's the question here?

> Don't you have bigger problems with file systems due to the crashes?

As it happens, no. Maybe because I'm using ext4.
Maybe I'm lucky.

> > On my system at least, it has no measureable cost,
> > likely also because size only looks at headers and metadata.
> 
> For example on OpenBSD, 'size' does not seem to come from binutils, so
> there could be portability issues.

True, I'm not saying it's perfect.

> >
> > If others are not interested, I can keep it out of tree.
> 
> I've had problems with disk close to full, so I'm semi-interested if
> the solution does not slow down others and it's not too ugly.

I think the simplest way to do it is to change makefile to unlike, create
then rename. Then you are safe against abrupt killing or crashing make.
And with a journaled fs, if you also have e.g. linux ext4 and mount with
data=ordered, you are safe against power failures.

It shouldn't be hard to do and I don't expect this to have any
measureable speed impact.  What do you think?

> >
> >> >>
> >> >> -- PMM
> >> >
> >> >