|
From: | Guillaume Le Vaillant |
Subject: | Re: When substitute download + decompression is CPU-bound |
Date: | Thu, 07 Jan 2021 11:45:21 +0100 |
User-agent: | mu4e 1.4.13; emacs 27.1 |
I compared gzip, lzip and zstd when compressing a 580 MB pack (therefore containing "subsitutes" for several packages) with different compression levels. Maybe the results can be of some use to someone. Note that the plots only show the results using only 1 thread and standard compression levels, and that the speed axis is using logarithmic scale.
Machine used for the tests: - CPU: Intel i7-3630QM - RAM: 16 MiB Programs: - gzip 1.10 - pigz 2.4 - lzip 1.21 - plzip 1.8 - zstd 1.4.4 - pzstd 1.4.4 Uncompressed file: - name: monero-0.17.1.5-pack.tar - size: 582707200 bytes #+PLOT: script:"compression-benchmark.plot" | Comp. command | Comp. time | Comp. size | Comp. speed | Comp. ratio | Decomp. time | Decomp. speed | |----------------+------------+------------+-------------+-------------+--------------+---------------| | gzip -1 | 7.999 | 166904534 | 72847506 | 3.491 | 3.292 | 50700041 | | gzip -2 | 8.469 | 161859128 | 68804723 | 3.600 | 3.214 | 50360650 | | gzip -3 | 10.239 | 157839772 | 56910558 | 3.692 | 3.144 | 50203490 | | gzip -4 | 11.035 | 151039457 | 52805365 | 3.858 | 3.104 | 48659619 | | gzip -5 | 13.767 | 146693142 | 42326375 | 3.972 | 3.143 | 46672969 | | gzip -6 | 19.707 | 144364588 | 29568539 | 4.036 | 3.001 | 48105494 | | gzip -7 | 24.014 | 143727357 | 24265312 | 4.054 | 2.993 | 48021168 | | gzip -8 | 43.219 | 143062985 | 13482663 | 4.073 | 2.969 | 48185579 | | gzip -9 | 70.930 | 142803637 | 8215243 | 4.080 | 2.964 | 48179365 | | pigz -1 -p 4 | 2.247 | 165745308 | 259326747 | 3.516 | 1.919 | 86370666 | | pigz -2 -p 4 | 2.394 | 160661935 | 243403175 | 3.627 | 1.862 | 86284605 | | pigz -3 -p 4 | 2.776 | 156696382 | 209908934 | 3.719 | 1.817 | 86239065 | | pigz -4 -p 4 | 3.045 | 150539955 | 191365255 | 3.871 | 1.787 | 84241721 | | pigz -5 -p 4 | 3.855 | 146289903 | 151156213 | 3.983 | 1.732 | 84462992 | | pigz -6 -p 4 | 5.378 | 143967093 | 108350167 | 4.048 | 1.721 | 83653163 | | pigz -7 -p 4 | 6.579 | 143350506 | 88570786 | 4.065 | 1.702 | 84224739 | | pigz -8 -p 4 | 11.76 | 142738270 | 49549932 | 4.082 | 1.720 | 82987366 | | pigz -9 -p 4 | 19.878 | 142479078 | 29314176 | 4.090 | 1.691 | 84257290 | | lzip -0 | 16.686 | 130302649 | 34921923 | 4.472 | 9.981 | 13055070 | | lzip -1 | 42.011 | 118070414 | 13870348 | 4.935 | 8.669 | 13619842 | | lzip -2 | 51.395 | 112769303 | 11337819 | 5.167 | 8.368 | 13476255 | | lzip -3 | 69.344 | 106182860 | 8403138 | 5.488 | 8.162 | 13009417 | | lzip -4 | 89.781 | 100072461 | 6490318 | 5.823 | 7.837 | 12769231 | | lzip -5 | 119.626 | 95033235 | 4871075 | 6.132 | 7.586 | 12527450 | | lzip -6 | 155.740 | 83063613 | 3741538 | 7.015 | 6.856 | 12115463 | | lzip -7 | 197.485 | 78596381 | 2950640 | 7.414 | 6.586 | 11933857 | | lzip -8 | 238.076 | 72885403 | 2447568 | 7.995 | 6.227 | 11704738 | | lzip -9 | 306.368 | 72279340 | 1901985 | 8.062 | 6.203 | 11652320 | | plzip -0 -n 4 | 4.821 | 131211238 | 120868533 | 4.441 | 2.829 | 46380784 | | plzip -1 -n 4 | 13.453 | 120565830 | 43314294 | 4.833 | 2.604 | 46300242 | | plzip -2 -n 4 | 15.695 | 114874773 | 37126932 | 5.073 | 2.398 | 47904409 | | plzip -3 -n 4 | 20.563 | 108896468 | 28337655 | 5.351 | 2.486 | 43803889 | | plzip -4 -n 4 | 26.871 | 102285879 | 21685356 | 5.697 | 2.375 | 43067739 | | plzip -5 -n 4 | 35.220 | 97402840 | 16544781 | 5.982 | 2.448 | 39788742 | | plzip -6 -n 4 | 45.812 | 89260273 | 12719532 | 6.528 | 2.145 | 41613181 | | plzip -7 -n 4 | 62.723 | 82944080 | 9290168 | 7.025 | 2.080 | 39876962 | | plzip -8 -n 4 | 71.928 | 78477272 | 8101257 | 7.425 | 2.120 | 37017581 | | plzip -9 -n 4 | 103.744 | 75648923 | 5616780 | 7.703 | 2.578 | 29344035 | | zstd -1 | 2.057 | 145784609 | 283280117 | 3.997 | 0.639 | 228144928 | | zstd -2 | 2.316 | 136049621 | 251600691 | 4.283 | 0.657 | 207077049 | | zstd -3 | 2.733 | 127702753 | 213211562 | 4.563 | 0.650 | 196465774 | | zstd -4 | 3.269 | 126224007 | 178252432 | 4.616 | 0.658 | 191829798 | | zstd -5 | 5.136 | 122024478 | 113455452 | 4.775 | 0.680 | 179447762 | | zstd -6 | 6.394 | 120035201 | 91133438 | 4.854 | 0.652 | 184103069 | | zstd -7 | 8.510 | 116048780 | 68473231 | 5.021 | 0.612 | 189622190 | | zstd -8 | 9.875 | 114821611 | 59008324 | 5.075 | 0.593 | 193628349 | | zstd -9 | 12.478 | 113868149 | 46698766 | 5.117 | 0.588 | 193653315 | | zstd -10 | 14.982 | 111113753 | 38893819 | 5.244 | 0.578 | 192238327 | | zstd -11 | 16.391 | 110674252 | 35550436 | 5.265 | 0.583 | 189835767 | | zstd -12 | 21.008 | 110031164 | 27737395 | 5.296 | 0.570 | 193037130 | | zstd -13 | 51.259 | 109262475 | 11367900 | 5.333 | 0.561 | 194763770 | | zstd -14 | 58.897 | 108632734 | 9893665 | 5.364 | 0.562 | 193296680 | | zstd -15 | 82.514 | 107956132 | 7061919 | 5.398 | 0.557 | 193817113 | | zstd -16 | 78.935 | 105533404 | 7382114 | 5.522 | 0.576 | 183217715 | | zstd -17 | 89.832 | 94165409 | 6486633 | 6.188 | 0.565 | 166664441 | | zstd -18 | 115.663 | 91124039 | 5037974 | 6.395 | 0.614 | 148410487 | | zstd -19 | 157.008 | 90229137 | 3711322 | 6.458 | 0.614 | 146952992 | | zstd -20 | 162.499 | 80742922 | 3585913 | 7.217 | 0.605 | 133459375 | | zstd -21 | 207.122 | 79619348 | 2813353 | 7.319 | 0.611 | 130309899 | | zstd -22 | 277.177 | 78652901 | 2102293 | 7.409 | 0.634 | 124058203 | | pzstd -1 -p 4 | 0.621 | 146665510 | 938336876 | 3.973 | 0.196 | 748293418 | | pzstd -2 -p 4 | 0.720 | 137416958 | 809315556 | 4.240 | 0.227 | 605361048 | | pzstd -3 -p 4 | 1.180 | 128748806 | 493819661 | 4.526 | 0.231 | 557354139 | | pzstd -4 -p 4 | 1.786 | 127373154 | 326263830 | 4.575 | 0.240 | 530721475 | | pzstd -5 -p 4 | 2.635 | 123216422 | 221141252 | 4.729 | 0.240 | 513401758 | | pzstd -6 -p 4 | 3.774 | 121257316 | 154400424 | 4.806 | 0.251 | 483096876 | | pzstd -7 -p 4 | 3.988 | 117361187 | 146115145 | 4.965 | 0.263 | 446240255 | | pzstd -8 -p 4 | 4.540 | 116172098 | 128349604 | 5.016 | 0.240 | 484050408 | | pzstd -9 -p 4 | 5.083 | 115237287 | 114638442 | 5.057 | 0.268 | 429989877 | | pzstd -10 -p 4 | 5.630 | 112359994 | 103500391 | 5.186 | 0.226 | 497168115 | | pzstd -11 -p 4 | 5.991 | 111969711 | 97263762 | 5.204 | 0.246 | 455161427 | | pzstd -12 -p 4 | 8.001 | 111326376 | 72829296 | 5.234 | 0.227 | 490424564 | | pzstd -13 -p 4 | 16.035 | 110525395 | 36339707 | 5.272 | 0.259 | 426738977 | | pzstd -14 -p 4 | 18.145 | 109957500 | 32113927 | 5.299 | 0.253 | 434614625 | | pzstd -15 -p 4 | 24.791 | 109358520 | 23504788 | 5.328 | 0.224 | 488207679 | | pzstd -16 -p 4 | 23.940 | 106888588 | 24340317 | 5.452 | 0.234 | 456788838 | | pzstd -17 -p 4 | 29.099 | 97393935 | 20024991 | 5.983 | 0.266 | 366142613 | | pzstd -18 -p 4 | 37.124 | 94273955 | 15696240 | 6.181 | 0.284 | 331950546 | | pzstd -19 -p 4 | 48.798 | 93531545 | 11941211 | 6.230 | 0.262 | 356990630 | | pzstd -20 -p 4 | 54.860 | 82067608 | 10621713 | 7.100 | 0.302 | 271747046 | | pzstd -21 -p 4 | 64.179 | 79735488 | 9079406 | 7.308 | 0.389 | 204975548 | | pzstd -22 -p 4 | 256.242 | 78688788 | 2274050 | 7.405 | 0.585 | 134510749 | #+TBLFM: $4='(format "%d" (round (/ 582707200.0 $2)));N :: $5='(format "%.3f" (/ 582707200.0 $3));N :: $7='(format "%d" (round (/ $3 $6)));N
compression-benchmark.plot
Description: Text document
compression.png
Description: PNG image
decompression.png
Description: PNG image
signature.asc
Description: PGP signature
[Prev in Thread] | Current Thread | [Next in Thread] |