Re: Feature to add

[Assaf Gordon]

Hello Russel,

On 19/07/18 06:27 AM, Russell Harper wrote:
> Thank you Assaf for the reply.
>
> Just another note with existing solutions in sed, awk, and perl, they 
> all seem to break if you want to process matches including $ or & 
> characters. Examples:

I'm not sure what do you mean by "break".
Do you mean that if the matched string has a special shell character 
(e.g. '$') then it is interpreted by the shell before being passed
to the program (e.g. to 'factor') ?

That will happen regardless of which language you are using - it's the 
way the shell works.

Perl has a simple mechanism to ensure variables are properly quoted
and not interpolated, like so:

  $ echo 'HOME $HOME $4.5 &77.66' \
       | perl -MString::ShellQuote -np \
              -e 'sub f($) { $v = shell_quote($_[0]);
                             $r = `echo ==$v==`; chomp $r ; $r ; }' \
              -e 's/(\S+)/f($1)/ge'
  ==HOME== ==$HOME== ==$4.5== ==&77.66==

Or even better, depending on your specific need, use perl's system()
with a LIST argument, and then the shell will not be used at all
(Perl will execute the program directly).

> Basically in all the existing solutions, two levels of quoting are 
> needed just for regular alphanumerics, and there's no room to process 
> arguments with $ or &.

If the execution of a sub-process goes through the shell, you'll need
to escape the parameters to avoid interpolation. That is true regardless
of which scripting language you use.

> A sed x flag for substitution would leave one 
> level of quoting to protect these arguments using single quotes:
>
> "/(\$[0-9]+\(\.[0-9]+\)?)/latest-quote '\\1' JPY/x"

In your example there are some assumptions that might not
be generic enough to apply to all cases:
1. You use double-quotes for the sed script, so you know you can safely
use single-quotes inside the pattern.
2. Your quoting does not protect against single-quotes (which is fine in
your specific case because the regex ensures there are no single quotes
in the matched pattern) - but this is not sufficient to safely cover all
cases.

I still think that the above Perl solution is an good one, generic and 
safe enough for most cases.


regards,
 - assaf