Re: how to sell network nodes

[Michael J. Baars]

On Fri, 2020-11-13 at 14:36 -0800, L A Walsh wrote:
> On 2020/11/12 00:48, Michael J. Baars wrote:
> > Hi,
> > 
> > I needed to zero out my hard drive because one of my nodes has become 
> > unstable. To this purpose I used coreutils dd with the following command 
> > line
> > arguments
> > 
> > dd if=/dev/zero of=/dev/mmc... status=progress
> > 
> > and I noticed how slow this program is in doing the job. So I tried a 
> > couple of different settings, like
> > 
> > bs=1048576 oflag=direct
> > 
> > but without significant improvement. The results are always the same... 
> > around 25 mb/s.
> > 
> > Then I remembered this little benchmark I write not so long ago, please do 
> > have a look at it, it won't destroy your drive. I included the results 
> > obtained
> > by running the benchmark on the computer I'm currently working on, so you 
> > can compare them to your own.
> >   
> ---
>     Your benchmark uses 'random' as input, which I seem to remember having
> it's own slowness.
> 
>     Using '/dev/zero' as input, I immediately got 49MB/s.

I get 83.2 mb/s using /dev/zero.

The actual input to the benchmark is neither /dev/zero or /dev/random, it's 
./ftesti :) Trying to achieve the same with dd:

dd if=./ftesti of=./ftesto bs=1048576 count=64 oflag=direct status=progress

gives a speed of 88.0 mb/s.

As you can see from the log, with given blocksize, my benchmark does this exact 
same thing at a rate of 5663.7168 mb/s and at a maximum rate of 16589.7124 mb/s
with blocksize 67108864 (the entire file at once).

Why, I ask you, is dd that much slower? What is it 'actually' doing with all 
the processing power available?

>     Ensuring the file was in 1 contiguous area on disk (used xfs_fsr) and
>      "nfrags"
>     I got     up to 56.1MB/s.
> 
>     Using a bs of 16MB and 4 blocks, I got up toe 64.3MB/s.
> 
>     Trying it on my larger RAID disk w/same sizes: 1.6MB/s.
> 
> 
> 
> > Hope that one of you feels inspired enough you to pick up the dd source 
> > codes and finish the job.
> >   
> ---
>     Is that what you meant?
> 
>     All of those are rotating rust...the 1st disk is a
> RAID5.  The larger Disk is a RAID10 with 6 stripes.
> 
>