[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[PATCH] Speedup batch replace using parallel find+sed

From: kitsunyan
Subject: [PATCH] Speedup batch replace using parallel find+sed
Date: Mon, 15 Feb 2021 10:27:50 +0300


I want to suggest my patch to improve makeicecat performance. The most
heavy operation script runs is batch replace, which involves find + sed
calls. It's possible to parallelize them using "xargs -P" instead of
relying on "find -execdir".

My patch introduces NPROC environment variable, which can be changed on
the user side (for instance, parsing MAKEFLAGS on Arch, or using
"multiprocessing" eclass on Gentoo, or simply prepending NPROC=`nproc`
before ./makeicecat call).

I decided to not calculate this parameter on the script side to allow
package maintainers to decide and change the variable according to
their distro-specific configuration (like MAKEFLAGS on Arch I
mentioned above).

On my dualcore SMT machine this change improved performance of
makeicecat script for around 3 times (from 20 minutes to 7 minutes).

I didn't inspect the script carefully, but the data in the output
tarball the same, so I assume this change won't break anything.

Attachment: 0001-Speedup-batch-replace-using-parallel-find-sed.patch
Description: Text Data

reply via email to

[Prev in Thread] Current Thread [Next in Thread]