bug#41563: Possible bug with 'sort -Vr' version sorting

[Erik Auerswald]

Hi,

On Thu, May 28, 2020 at 01:01:05PM +0200, Kamil Dudka wrote:
> On Thursday, May 28, 2020 11:02:43 AM CEST Erik Auerswald wrote:
> > On Thu, May 28, 2020 at 08:48:16AM +0200, Kamil Dudka wrote:
> > > It is the underscore in the .x86_64 suffix what breaks the version compare
> > > algorithm.  If you replace the underscore by an alphabetic character, it
> > > sorts as you expect:
> > > 
> > > # ls -t /boot/vmlinuz-* | sed "s/\/boot\/vmlinuz-//g" | grep -v rescue | \
> > > 
> > >     sed 's/x86_64/x86X64/' | sort -Vr | sed 's/x86X64/x86_64/'
> > > 
> > > 3.10.0-1127.8.2.el7.x86_64
> > > 3.10.0-1127.el7.x86_64
> > > 3.10.0-1062.18.1.el7.x86_64
> > 
> > That is interesting.  The underscore can be replaced by a digit or even
> > removed as well.  Replacing it with a dot (.)  does not help.
> 
> If there is no underscore, the .el7.x86X64 suffix is recognized as file
> extension.  See the corresponding documentation:
> 
> https://www.gnu.org/software/coreutils/manual/html_node/Special-handling-of-file-extensions.html

Ah, el7.x86X64 or el7.x86164 is seen as an extension (i.e., a sequence
of suffixes), but el7.x86.64 or el7.x86_64 is not.  Since .8.2 does not
contain a letter, it is not seen as part of the extension.  Very subtle,
but documented.

Trvia: the usual 7-Zip extension .7z is no suffix resp. file extension
for this algorithm (according to the documented definition).

Thus changing the platform indicator to look like a file extension,
and relying on the behavior that the distribution version information
is interpreted as a file extension as well, you create a file extension
where initially there was none.  This file extension is then ignored for
the comparison, unless that comparison results in equality.  This seems
to be a useful hack when working with Red Hat products.

Fascinating. :-)

> > This differs from Debian's "dpkg --compare-versions", where the results
> > of the comparison do not change by replacing the underscore with a
> > digit or character, or by removing it (the underscore is identified as
> > problematic, though):
> 
> The problem is that `dpkg --compare-versions` expects version numbers only.
> It does not work well if you feed it with file names including extensions:

I did not, as you can see in the examples.  I gave version information
to dpkg, though not Debian version information.  So of course this is
illegal input and the GIGO principle applies.

> $ dpkg --compare-versions 3.10.0-1127.8.2 '>>' 3.10.0-1127 && echo '>>' || 
> echo '<='
> >>
> $ dpkg --compare-versions 3.10.0-1127.8.2.bz2 '>>' 3.10.0-1127.bz2 && echo 
> '>>' || echo '<='
> <=
> 
> >     $ dpkg --compare-versions 3.10.0-1127.8.2.el7.x86_64 lt
> > 3.10.0-1127.el7.x86_64 && echo less dpkg: warning: version
> > '3.10.0-1127.8.2.el7.x86_64' has bad syntax: invalid character in revision
> > number dpkg: warning: version '3.10.0-1127.el7.x86_64' has bad syntax:
> > invalid character in revision number less
> >     $ dpkg --compare-versions 3.10.0-1127.8.2.el7.x86.64 lt
> > 3.10.0-1127.el7.x86.64 && echo less less
> >     $ dpkg --compare-versions 3.10.0-1127.8.2.el7.x86X64 lt
> > 3.10.0-1127.el7.x86X64 && echo less less
> >     $ dpkg --compare-versions 3.10.0-1127.8.2.el7.x86164 lt
> > 3.10.0-1127.el7.x86164 && echo less less
> >     $ dpkg --compare-versions 3.10.0-1127.8.2.el7.x8664 lt
> > 3.10.0-1127.el7.x8664 && echo less less
> > 
> > The way I read the GNU Coreutils documentation, removing the underscore
> > should not affect the version sort comparison result.
> 
> Not really.  See the link above to the documentation that covers this part.

Yes, you are correct.  I find this quite surprising, and see it as another
example where --version-sort fails to deliver on the short form promise
of "natural sort."  I am well aware that the long form description shows
that the sorting order is not "natural," but rather strange IMHO.

    $ sort --help | grep -- --version-sort
      -V, --version-sort          natural sort of (version) numbers within text

But then I do not even understand what is "natural" about version numbers
anyway. ;-)

Thanks,
Erik
-- 
[M]ost parts of this industry just work by chance.
                        -- Thomas Gleixner