Re: [PATCH] wc: speed-up by simplifying avx code

[Pádraig Brady]

On 31/03/2024 18:58, Evgeny Nizhibitsky wrote:
> Yes, it's true that simplifying and speeding-up by the bufsize increase are two 
> different things although the former allowed the latter.
>
> I just landed more tests with hyperfine for various configurations spanning over the current master 
> version and a new approach with a range of bufsizes from 16 KiB up to 1 MiB, running on 1 billion 
> yes'es like you did (1by), a generated file for the recent 1 billion row challenge (1brc, with 
> entries like "<station name>;<temperature:0.2f>") and the first 100 million 
> rows for both of them (100my and 100mrc, respectively), all in /dev/shm, yet again with 7800X3D:
>
> The reported timings are as follows:
>
> | version | 100my | 100mrc | 1by | 1brc |
> | ------- | ------- | ------- | ------- | ------- |
> | master | 21.3 ms ± 1.0 ms | 163.1 ms ± 1.5 ms | 197.1 ms ± 3.0 ms | 1.680 s ± 
>  0.010 s |
> | 16 KiB | 21.0 ms ± 1.1 ms | 163.7 ms ± 2.1 ms | 194.3 ms ± 2.5 ms | 1.658 s ± 
> 0.015 s |
> | 32 KiB | 20.2 ms ± 0.7 ms | 158.9 ms ± 3.0 ms | 194.6 ms ± 6.4 ms | 1.620 s ± 
> 0.023 s |
> | 64 KiB | 19.8 ms ± 0.6 ms | 154.0 ms ± 5.3 ms | 187.5 ms ± 7.2 ms | 1.553 s ± 
> 0.013 s |
> | 128 KiB | 18.8 ms ± 0.6 ms | 148.9 ms ± 5.4 ms | 178.4 ms ± 1.3 ms | 1.530 s 
> ± 0.013 s |
> | 256 KiB | 19.2 ms ± 0.8 ms | 145.8 ms ± 1.5 ms | 176.4 ms ± 1.6 ms | 1.522 s 
> ± 0.017 s |
> | 512 KiB | 19.6 ms ± 0.7 ms | 146.4 ms ± 1.0 ms | 183.0 ms ± 5.0 ms | 1.512 s 
> ± 0.014 s |
> | 1 MiB | 19.3 ms ± 0.7 ms | 145.7 ms ± 1.8 ms | 188.4 ms ± 6.2 ms | 1.499 s ± 
> 0.012 s |
>
> And the corresponding speed-up values are as follows:
>
> | version | 100my | 100mrc | 1by | 1brc |
> | ------- | ------- | ------- | ------- | ------- |
> | master | 0% | 0% | 0% | 0% |
> | 16 KiB | 1% | 0% | 1% | 1% |
> | 32 KiB | 5% | 3% | 1% | 4% |
> | 64 KiB | 8% | 6% | 5% | 8% |
> | 128 KiB | 13% | 10% | 10% | 10% |
> | 256 KiB | 11% | 12% | 12% | 10% |
> | 512 KiB | 9% | 11% | 8% | 11% |
> | 1 MiB | 10% | 12% | 5% | 12% |
>
> So again in my case the new approach is on par with the old one while the sweet 
> spot bufsize of 256 KB seems to bring the best value.
>
> Still more testing on different CPUs and sample files should probably be 
> conducted.

Excellent.
This concurs with my testing with this patch on my laptop,
and my testing of 256KiB buffer sizes with:
https://github.com/coreutils/coreutils/commit/fcfba90d0

I'll test on a few other systems and adjust the configure check before 
committing.

thanks!
Pádraig