Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT

[Ralf Wildenhues]

Hi Jim,

* Jim Meyering wrote on Sun, Oct 03, 2010 at 02:56:21PM CEST:
> Ralf Wildenhues wrote:
> > * Jim Meyering wrote on Sun, Oct 03, 2010 at 12:35:57PM CEST:
> >> * lib/am/distdir.am (dist-xz): Do not hard-code xz's -9: that
> >> made it impossible to override.  Instead, use its XZ_OPT envvar,
> >> defaulting to -9 if not defined.  Thus no change in behavior
> >> when XZ_OPT is not set, and now, this rule honors the setting
> >> of that envvar when it is set.  Suggested by Lasse Collin.
> >
> > Thanks.  A more general fix for all dist-* compressors, with hopefully
> > as uniform naming scheme as is possible throughout, and autoconf.texi

> automake.texi, I presume.

Of course; thanks.

> > and NEWS update, was already on my list of things to do.  (There are
> > some prior bug reports too.)
> 
> The name I used there, XZ_OPT, is dictated by the tool.  An alternative
> is to invoke xz with no explicit XZ_OPT setting, but that would result
> in a semantic change: using the default compression level of -7 rather
> than what was hard-coded as -9.  Perhaps that would be better, because
> some people have complained that using -9 makes it harder to decompress
> (requires more memory) on extremely low-memory (embedded) systems.

The change to use -9 for xz is fairly recent, new in 1.11.1.

> Hence, my using an explicit XZ_OPT setting served three purposes:
> - preserved the existing -9 (perhaps not desirable)
> - allowed an override
> - documented in the Makefile that xz's behavior can be overridden
>     through that envvar.

Sounds good.

> There is no point in changing lzma, since it's deprecated.

Except that the legacy lzma Utils that I tested on one MSYS install
failed with OOM due to the -9, causing a spurious testsuite failure.

Wrt. xz: I think the general approach for 'dist-*' should be: by default
use highest compression that is portable.  Rationale: the default
behavior is one compression plus upload, and lots of downloads and
decompressions.  That it makes testing the package take longer is a QoI
issue which the developer can tune with the XZ_OPT knob then.  OTOH,
when we find out that -9 fails due to OOM on some systems (and I have no
idea whether the lzma utils failure corresponds to a similar xz failure)
then we should consider backing down to something lower.  I really
prefer the defaults to work OOTB.

> bzip2 and lzip hard-code the -9, just like xz does/did.
> bzip2 honors BZIP2, but lzip appears to honor no envvar.
> gzip doesn't hard-code a compression level and besides,
> it already honors the $(GZIP_ENV) makefile variable.
> 
> With all that, trying to make an across-the-board, consistent change
> appears to be a lost cause.

:-/

> As a half measure, I could change bzip2 in the same way as xz, but
> hesitated to do even that, since I feel that bzip2 has been subsumed.
> Tell me what you'd like.

I would be fine with an analogous bzip2 change.

If you see chance for testsuite additions anywhere and have time for it,
all the better.  Sorry I'm a bit out of time right now.

I'm not quite sure if I want to see all these changes in branch-1.11 any
more though, but am certainly open to input/comments here.

Thanks,
Ralf