bug#17470: [PATCH] sort: rotate on ENOSPC while creating tmp files

[Azat Khuzhin]

On Wed, May 14, 2014 at 07:48:39AM -0700, Paul Eggert wrote:
> Pádraig Brady wrote:
> >Also do we always know how much to fallocate?
> 
> Not if we're using compression on the temporaries, no.
> 
> I think a patch along these lines could be worthwhile, if it was simple and
> if it actually worked (the current one doesn't).  

The current patch only look while files is created, but this is not
enough `I agree with you, it must check write(2) and fallback to creating
when write(2) will fail with ENOSPC.
This is what you mean?

> Something along the
> following lines, say.  When multiple -T options are specified (-T FOO, -T
> FOP, -T FOQ, ...) and one of them runs out of disk space when creating a
> temporary file FOO/BAR, 'sort' stops creating files in FOO (effectively
> removing FOO from the option list) creates a file FOP/BAR instead, and
> redoes the process (whatever it was) that sent output to FOO/BAR, sending
> the output to FOP/BAR this time.

I don't think that redoes is worth it, since when we have ENOSPC it
means that we already won't create any more files there, and one file
with relatively small size is not a big deal.
I also think that dropping direcotory from list is a good idea, user can
notice this (using some monitorings) and clean it.

For example recently I need to sort relatively huge amount of data, and
I don't have enough space for writing all tmp files (why
--compress-program not works for me is another story), so I wrote
script, that did next:
When free space < %3 on some of partitions, I run pkill -STOP sort, and
then move some files (that not currently opened by sort(1)) to partition
that have more free space, and create symlinks for old locations.
When all the free space was eliminated on all available partitions, I
archived existed tmp files (that also not opened by sort(1)) and create
a pipes, that redirects output of $(gzip -d) into it, and using this
hacks sort finished successfully for me.


> 
> I don't have the energy right now to write that, but if someone else wrote
> it I'd review it.

Thanks for you notes Pádraig.

-- 
Respectfully
Azat Khuzhin