coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

seq + factor improvements


From: Pádraig Brady
Subject: seq + factor improvements
Date: Wed, 24 Jun 2015 06:21:37 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0

I was testing with a naïve¹ prime sieve to check
seq, factor and split --filter parallel functionality, using:

  odd() { LC_ALL=C sed '/[24680]$/d'; }
  primes() { LC_ALL=C sed 's/.*: //; / /d'; }
  seq inf | odd | split -u -nr/$(nproc) --filter='factor' | primes

Two problems with the above are that there is a
format change in seq in the output from 999999 to 1e+06, and
factor output is corrupted due to interspersed lines.

The attached patch set fixes those two issues.
Also included are improvements for related issues in seq.

For completeness, here are some benchmarks with the fixed up tools:

# 500 large numbers are factored faster in parallel
# (using large constants to avoid inaccurate floating point conversion)
# (using -u option to split keeps the factor processes supplied).
  $ time seq 10000000000000000000000000000000 10000000000000000000000000001000 |
      odd | factor | primes | wc -l
  12
  real    0m21.188s
  user    0m21.200s
  sys     0m0.014s

  $ time seq 10000000000000000000000000000000 10000000000000000000000000001000 |
      odd | split -u -nr/$(nproc) --filter='factor' | primes | wc -l
  12
  real    0m11.949s
  user    0m24.961s
  sys     0m0.019s


# 5 million small numbers are factored in about the same time in parallel
# (now without split -u option to improve batching of quickly processed items).
  $ time seq 1e7 | odd | factor | primes | wc -l
  664579
  real    0m7.165s
  user    0m16.023s
  sys     0m0.328s

  $ time seq 1e7 | odd | split -nr/$(nproc) --filter='factor' | primes | wc -l
  664579
  real    0m7.189s
  user    0m15.367s
  sys     0m0.447s

cheers,
Pádraig

¹ http://primesieve.org/

Attachment: factor-parallel.patch
Description: Text Data


reply via email to

[Prev in Thread] Current Thread [Next in Thread]