Re: RFC: Safely using xargs -P$NUM children's output? Need a new tool?

[Denys Vlasenko]

On 5/2/19 8:25 PM, Egmont Koblinger wrote:
> Hi Denys,
>
> (Disclaimer: I'm just a random guy commenting here, not affiliated
> with the coreutils team.)
>
> The simple solution recommended by xargs's man page at the -P option
> is to use separate output file for each parallel "file".
>
> One concern with your suggested coalesce approach is that this tool
> itself also needs to write its output in an atomic step. If coalesce
> itself performs a partial write then you're back to square one.

That's the idea: the tool would be defined to do a single write.
(Unlike e.g. "tail -nNUM", which you might expect to do one write
when it's done, but in fact it does multiple writes).

> I guess (and that's what you also say) if the output goes to a local
> file system then complete writes are guaranteed. Might depend on the
> file system type or other factors though, a kernel guru could clarify> it. I'm 
> not so sure about virtual, remote, fuse etc. file systems.

I do remember that specifically writes to filesystem (as
opposed to writes to file descriptors in general) are not allowed
to become partial randomly. It was discussed on LKML.