Re: Bug in version 20170322: certain functions in the Bash environment break invocation of env_parallel on macOS 10.12.4

[Michael Klement]

Thanks for the detailed response.

Yes, a specific error message might be helpful.

I can confirm that the --env approach is effective as a workaround in my case.

> As the above is the recommended usage, maybe --env _ should be default
> if ~/.parallel/ignored-vars exist?


That might be a little obscure.

I think if the environment problem can be reliably detected, then the 
associated error message could recommend the --env approach as an alternative.

Michael

> On Apr 16, 2017, at 8:57 AM, Ole Tange <ole@tange.dk> wrote:
> 
> On Sat, Apr 15, 2017 at 3:06 PM, Michael Klement
> <michael.klement@usa.net> wrote:
> 
>> I'm aware of that workaround, but I thought you'd want to know that
>> env_parallel is not fully robust and fails in obscure ways.
>> Is env_parallel deprecated? The man page doesn't seem to say so.
> 
> env_parallel is pretty new and thus not the same production quality as 
> parallel.
> 
>> 2017-04-14 20:53 GMT+02:00 Michael Klement <michael.klement@usa.net>:
> :
>>> v3.2.57: only stderr output:
>>> 
>>> -bash: /usr/bin/which: Argument list too long
>>> -bash: -n: command not found
> 
> Given your attached file, you are most likely being hit by the
> environment being too big:
> 
> From man env_parallel:
> 
>   Environment space
>      By default env_parallel will export all environment
>      variables, arrays, aliases, functions and shell options
>      (see details for the individual shells below).
> 
>      But this only works if the size of the current environment
>      is smaller than the maximal length of a command and
>      smaller than half of the max if running remotely. E.g. The
>      max size of Bash's command is 128 KB, so env_parallel will
>      fail if set | wc -c is bigger than 128 KB. Technically the
>      limit is in execve(1) which IPC::open3 uses.
> 
>      Bash completion functions are well-known for taking up
>      well over 128 KB of environment space and the primary
>      reason for causing env_parallel to fail.
> 
>      Instead you can use --env to specify which variables,
>      arrays, aliases and functions to export as this will only
>      export those with the given name.
> 
> And earlier:
> 
>      Due to the problem with environment space (see below)
>      the recommended usage is:
> 
>        # Run once to record the "empty" environment
>        env_parallel --record-env
>        alias myalias=echo
>        myvar=it
>        myfunc() { myalias $1 $myvar works.; }
> 
>        # Use --env _ to only transfer the names not in the "empty" environment
>        env_parallel --env _ -S localhost myfunc ::: Yay,
> 
> But even tough it is documented your bug report is valuable:
> env_parallel should detect this error and give you a sensible error
> message instead.
> 
> As the above is the recommended usage, maybe --env _ should be default
> if ~/.parallel/ignored-vars exist?
> 
> 
> /Ole