[Mingw-cross-env-list] New features: automatic "du", automatic log compression

[Volker Grabsch]

Dear mingw-cross-env people,

I just added a new feature to mingw-cross-env so that "du"
is called immediately after each successful build of a
package. It should also work on BSD systems.

    http://hg.savannah.gnu.org/hgweb/mingw-cross-env/rev/46c4a650912a

That way, you can easily gather statistical data by grepping
over the log files:

    grep -H ^du: log/*

I'm using this to have a better estimation about how much disk
space a full build needs temporarily.

By sorting the result (numerically, by second column) you can
easily produce a top ranking about the biggest builds:

    grep -H ^du: log/* | sort -rnk2

On my system (Debian/Testing on amd64), this produces:

    log/wxwidgets:du: 1737716 KiB
    log/gcc:du: 1551720 KiB
    log/qt:du: 1260152 KiB
    log/boost:du: 907424 KiB
    log/libgomp:du: 639252 KiB
    ...

While investigating file sizes, I noticed that the build logs
became an important factor. A successful build produces about
97 MiB of log data! Compressing those with "gzip -9" makes them
implode to 4.9 MiB. Using bzip2 they become even smaller (3.8 MiB).

So I'm thinking about compressing the log files on the fly.

This would keep the logs small. It would also solve the
problem that people accidently post uncompressed logs to this
list, triggering the maximum attachment size. Statistical
analysis would still be possible using zgrep or bzgrep, e.g.

    zgrep -H ^du: log/*.gz | sort -rnk2
    bzgrep -H ^du: log/*.bz2 | sort -rnk2

So what do you think about this?

If we do that, should we use gzip or bzip2? gzip decompresses
a lot faster, but bzip2 will make the logs grow about 22%
slower. So I guess this is about archiving vs. analyzing.

I'd love to hear some more opinions / ideas about this.


Greets,
Volker

-- 
Volker Grabsch
---<<(())>>---