Re: does parallel handle --null and --block correctly?

[Ecks Hecker]

Am 09.03.2024 um 20:17 schrieb Ole Tange:
> Yep. --null is another way to write --delimiter '\0'. It is different
> from --recend '\0'.
>
> --delimiter is rarely used with --pipe, and --recend is always used
> with --pipe/--pipepart.
>
> Here is an example where -d is used with --pipe:
>
> $ seq 330000 | parallel --tag --pipe -d , wc {} ::: -l,-w,-c
> -l      165668
> -c      101752
> -w      149796
>
> In other words: It is not a bug. Instead it is a feature of limited use.
>
> You have not provided us with the file 'source' so we do not know what
> you are trying to do.
>
> My guess is that you are looking for --recend '\0'
>
> perl -e 'print map { "$_\0" } 1..1000000' | parallel --recend '\0' --pipe wc
>
> If that is the case please repay by explaining what should be changed
> in the documentation so it would have been obvious to you.
>
>
> /Ole
Thank you for initiating further investigations from my side, as i did
abandon my project at the time.
English not being my first language, i thought, that --null and --recend
'\0' and -d '\0' would all mean the same thing, thus my surprise, that
--pipe was not handled the way, i did expect.

But your example proved, that --recend yould indeed make parallel do
what i want.
In my example, the file named "source" was just the output from some
"find --print0" command, that i caught in order to play with a pipe for
a while. And resuming this play/investigation, i found, that i can solve
my initial problem (of distributing work of an enormous amount of files
onto the processor cores in parallel) just fine.

To meet your request, i have been rereading the man pages attentively. I
found them to be clear and concise enough. As i am using the --pipe
construct for the first time i was very much interested in SPREADING
BLOCKS OF DATA.
But my confusion concerning --null, --delimiter and --recend was not
based on bad documentation, but on my failure to notice, that recend
only concerns --pipe, whereas the other 2 do not apply in that context.
Can you construct a case, where both could be useful? I fail to imagine
one, and thus thought, they were identical.

I still like (and use) parallel quite a bit, especially due to its
versatility and ease of use compared to xargs or bash.
Good job creating and evolving it!
/Ecks