Re: sort dynamic linking overhead

[Pádraig Brady]

On 26/02/2024 06:25, Paul Eggert wrote:
> On 2023-10-09 06:48, Pádraig Brady wrote:
>
> > An incremental patch attached to use xxhash128 (0.8.2)
> > shows a good improvement (note avx2 being used on this cpu):
>
> xxhash128 is not a cryptographic hash function, so it doesn't attempt to
> be random. Of course most people won't care - it's random "enough" - but
> it would be a functionality change.
>
> blake2 is cryptographic and would be random, but would bloat the 'sort'
> executable with code that's hardly ever used.
>
> To attack the problem in a more conservative way, I installed the
> attached patch into coreutils. With it, 'sort -R' continues to use MD5
> but on GNUish platforms 'sort' links libcrypto dynamically only if -R is
> used (Bruno's suggestion). This doesn't significantly affect 'sort -R'
> performance, and reduces the startup overhead of plain 'sort' to be what
> it was before we started passing -lcrypto to gcc by default (in
> coreutils 8.32).
>
> I also toyed with changing MD5 to SHA512, but that hurt performance. For
> what it's worth, although I tested with an Intel Xeon W-1350, which
> supports SHA-NI as well as various AVX-512 options, I didn't see where
> libcrypto (at least on Ubuntu 23.10, which has OpenSSL 3.0.10) takes
> advantage of these special-purpose instructions.

I noticed that the dlopen() used an unversioned lib,
which generally won't work as that's what's available
at build time, not at run time.
Attached is a fix.
Note that also includes a comment for why we
check for SHA512 and not MD5 (to avoid deprecation warnings).

Also attached is a fix to a sc_tight_scope failure.

cheers,
Pádraig

0001-maint-avoid-sc_tight_scope-failure-in-sort.c.patch
Description: Text Data

0002-build-fix-libcrypto-version-linked-by-sort-at-runtim.patch
Description: Text Data