Re: [PATCH] sort: add --threads option to parallelize internal sort.

[Chen Guo]

Hi Padraig,
    Thanks for looking everything over.


> option.
I've renamed that to --parallel in the attached patch
(along with 
> other tweaks and cleanups). What do you think?
You're the maintainer :-)

I still have a couple 
> questions on the option though.
1. Is limiting the number to that returned by 
> nproc()
always the right thing todo?
The reason for this was because we use spinlocks,
and if we try to run on more threads than cores, and
two threads on the same core want to grab the same
lock, this could lead to quite a bit of unnecessary
spinning. We chose spinlocks because it was slightly
faster than mutexes.

A reason to switch to mutexes in the future might be
if technologies like hyperthreading (i.e. more logical
cores than physical cores) dominate, in which case
it would also be appropriate to remove the cap on
nthreads.

2. How does this option fit with the 
> threaded external sort.
I.E. the integrated patches will need to share out 
> the
threads to use, and perhaps multiple per CPU is 
> appropriate,
especially in the presence of files on slow 
> devices.
Yeah that is a good point, a lot of work can be
done while waiting for I/O. I dont remember off the
top of my head how much slower running with
mutexes in struct merge_node, but I'll test it out
and with the other patch (Joey merged the two
last night) in a few hours and see what's up.

Btw, from the memcoll thread, heh I never would've
thought of alignemnt.
The only thing I was questioning was the --threads