Re: [Help-bash] indirection for arrays

[Dan Douglas]

On Saturday, May 05, 2012 02:16:36 PM Bill Gradwohl wrote:

> Via a web site reference, Dan Douglas (Thank You) provided something akin to
> this:
> 
> 
> func() {
>         local -a 'keys=("${!'"$1"'address@hidden")' 
>         ...
>         ...
> }
> 
> declare -a myArray=(1 2 3)
> 
> func 'myArray'
> 
> 
> 
> Neat trick! Took me a while to figure out how it works. The single quotes in
> particular stopped me in my tracks for a while.

Yeah, after ages of trying to figure out a few quirks in compound assignment
that appeared specific to declare, I came up with that. The only related
reference to this I've been able to find since then is:
http://lists.gnu.org/archive/html/bug-bash/2004-09/msg00135.html

>From what I can tell, parsing of compound assignment arguments to declaration
commands is dynamically determined according to how things are quoted such that
only one pass of expansion is ordinarily allowed. If Bash thinks a bare
unquoted assignment is given, then it only allows expansions to occur as
arguments to declare, otherwise the expansions are performed by declare itself,
similarly to eval, except the result is assigned to an array rather than
treated as a command.

 # "$x" expanded as arg
 $ unset -v x; x=foo declare -a arr=("$x"); declare -p arr
 declare -a arr='()'

 # declare expanding "$x"
 $ unset -v x; x=foo declare -a 'arr=("$x")'; declare -p arr
 declare -a arr='([0]="foo")'

However, it's possible to trick declare into both letting pre-expanded results
through and performing expansions itself. The trigger (aside from a few
corner-cases I haven't figured out yet) is to quote or escape the parentheses
of the assignment: `arr=\(...\)'. Additionally, the variable must either
already have been given an array attribute by a previous command, or "-a" or
"-A" must be given explicitly, which forces handling as an array rather than
using some heuristic to determine what kind of assignment it is.

There are a huge number of useful applications for this. declare is much safer
and easier to use than eval because certain "printf %q-like" quoting is handled
automatically, and there's no case I've discovered so far in which the result
can be inadvertently treated as a command.

Here's my "random string" function as another example (which additionally
exploits a quirk in the way array parameters are parsed):

 # https://github.com/ormaaj/dotfiles/blob/master/bashrc#L215
 rndstr() {
     [[ $1 == +([[:digit:]]) ]] || return 1
     (( $1 )) || return
     local -a 'a=( {a..z} {A..Z} {0..9} )' \
              'b=( "address@hidden"{1..'"$1"'}"}" )'
     printf '%s' "address@hidden"
 }

> Can I rely on this in future versions of bash, or is this a side effect that
> might disappear some day? I can find no documentation that says specifically
> it's supported.
 
Based upon Chet's message above I'd say probably not. My testing shows this
working in Bash 3 though.

> Also, since indirection requires the ! immediately after a {, this means
> indirection can never be on the left hand side of an assignment. Correct?

It's been known for some time that expansions on the LHS work as you'd expect
(enables something like printf -v, only more backwards-compatible). That's a
different concept which doesn't rely on this quirk, and works for any type of
assignment - not only arrays.

> The "trick" above is doing a type of indirection by getting the parser to
> cooperate in a sneaky fashion. At least that's my analysis.
> 
> Is there some other mechanism ("trick") besides eval that can do indirection
> on the left hand side of an assignment? A mechanism that can be relied on in
> future releases.
> 
> -- Bill Gradwohl

I'm working on a library which abstracts this. My goal is to provide several
ways of passing and returning arrays, to enable something like ksh93's print -C
and read -C, and some simple functional programming niceties (like things from
the Python itertools). As a preview, here's one of the basic mechanisms:

 arrayBuilder() {
     set -- "$@" "${2}[*]" "address@hidden"
     local -a 'keys=("${!'"$1"'address@hidden")'
     local -a "$2"='( 
[\${keys[n]}]="\${'"$1"'[keys[n++]]"{1..'address@hidden'}"}" )'
     local -a "$2"='('"${!3}"')'
     declare -p "$2"
     printf '<%s> ' "${!4}"; echo
 }
 
 main() {
     local -a fromArr=(4 [2]=3 5 6 [11]='9 10' 14 [15]=$'15[ \'-6 *')
     arrayBuilder fromArr toArray
 }
 
 main

It's ugly and hard to read (and making it robust will be even worse) but that
should just be a few lines like this that do the dirty work. The "print
-C" I'm having some trouble implementing without parsing the result or passing
each element through printf %q individually and hoping it works, but I have some
ideas...  Getting arrays out of functions is much harder than getting them in.

-- Dan Douglas

signature.asc
Description: This is a digitally signed message part.