[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
seq + factor improvements
From: |
Pádraig Brady |
Subject: |
seq + factor improvements |
Date: |
Wed, 24 Jun 2015 06:21:37 +0100 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 |
I was testing with a naïve¹ prime sieve to check
seq, factor and split --filter parallel functionality, using:
odd() { LC_ALL=C sed '/[24680]$/d'; }
primes() { LC_ALL=C sed 's/.*: //; / /d'; }
seq inf | odd | split -u -nr/$(nproc) --filter='factor' | primes
Two problems with the above are that there is a
format change in seq in the output from 999999 to 1e+06, and
factor output is corrupted due to interspersed lines.
The attached patch set fixes those two issues.
Also included are improvements for related issues in seq.
For completeness, here are some benchmarks with the fixed up tools:
# 500 large numbers are factored faster in parallel
# (using large constants to avoid inaccurate floating point conversion)
# (using -u option to split keeps the factor processes supplied).
$ time seq 10000000000000000000000000000000 10000000000000000000000000001000 |
odd | factor | primes | wc -l
12
real 0m21.188s
user 0m21.200s
sys 0m0.014s
$ time seq 10000000000000000000000000000000 10000000000000000000000000001000 |
odd | split -u -nr/$(nproc) --filter='factor' | primes | wc -l
12
real 0m11.949s
user 0m24.961s
sys 0m0.019s
# 5 million small numbers are factored in about the same time in parallel
# (now without split -u option to improve batching of quickly processed items).
$ time seq 1e7 | odd | factor | primes | wc -l
664579
real 0m7.165s
user 0m16.023s
sys 0m0.328s
$ time seq 1e7 | odd | split -nr/$(nproc) --filter='factor' | primes | wc -l
664579
real 0m7.189s
user 0m15.367s
sys 0m0.447s
cheers,
Pádraig
¹ http://primesieve.org/
factor-parallel.patch
Description: Text Data
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- seq + factor improvements,
Pádraig Brady <=