Re: bug-bash Digest, Vol 238, Issue 2

[Yair Lenga]

Martin brings up several good points, and I think it's worth figuring out
the direction of the implementation. Bash currently does not have good
syntax for H-values, so a new one is needed. It does not make sense to have
a completely new one, as there are few accepted syntax - python,
JavaScript, Perl, Java to name a few. Ideally, bash will decide on a
direction, and then apply bash-style changes.

Wanted to emphasize - those are my own preferences. There is NO right
answer here.

Personally, I think adopting Javascript/Python like approach (${a.b.c} ) is
preferred over using Perl approach ( ${a{b}{c}} ), or sticking with the
existing bash approach. The main reason is that it is better to have a
syntax similar/compatible with current/future directions, and not the past.
JavaScript/Python knowledge is much more common nowadays vs Perl or Bash
arrays (regular, and associative). Using '.' is also in line with many
scripting/compiled languages - C/C++/Java/Groovy/lua - all support '.' for
choosing fields from structure - developers with this background will feel
comfortable with this grammar.

I believe supporting bracket notation, can be a plus. One tricky issue - Is
the content inside the bracket (${a[index_expression}} is evaluated or not.
Martin highlight some background compatibility issues, My vote is going to
evaluating the expressions.

Bottom line - IHMO - no point in joining a losing camp (Perl), or having a
"bash" camp.

That brings the second question of bash adaptations. My own opinion is that
it will be great to support multiple approaches:
* a.$b.$c - bash style substitution, should include ANY substitution,
including command, arithmetic, ...
* a[expr1][expr2] - will be nice alternative, where b and c are
evaluated/interpolated.
* a.$b[$expr] - mixed usage ?

As far as supporting '.', in the key, I do not see this as a major issue.
For me, the main goal is to support light-weight structure-like values. Not
deep hash tables. Bash will not be the preferred solution for complex
processing of data structure, even if it will be able to support '.' in the
key.

Needless to say - those are my own preferences. There is NO right answer
here.

Yair

On Mon, Sep 5, 2022 at 4:15 AM Martin D Kealey <martin@kurahaupo.gen.nz>
wrote:

> Rather than var[i1.i2.i3] I suggest a C-like var[i1][i2][i3] as that
> avoids ambiguity for associative arrays whose keys might include ".", and
> makes it simpler to add floating point arithmetic later.
>
> I would like to allow space in the syntax to (eventually) distinguish
> between an object with a fairly fixed set of fields and a map with an
> arbitrary set of keys. Any C string - including the empty string - should
> be a valid key, but a field name should have the same structure as a
> variable name.
>
> Moreover I'm not so keen on ${var.$key}; I would rather switch the
> preferred syntax for associative arrays (maps) to a Perl-like ${var{key}}
> so that it's clear from the syntax that arithmetic evaluation should not
> occur.
>
> Then we can write ${var[index_expression].field{$key}:-$default}.
>
> Retaining var[key] for associative arrays would be one of the backwards
> compatibility options that's only available for old-style (single-level)
> lookups.
>
> These might seem like frivolous amendments, but they deeply affect future
> features; I can only highly a few things here.
>
> Taken together they enable expression parsing to be a continuation of the
> rest of the parser, rather than a separate subsystem that has to be
> switched into and out of, and so bash -n will be able to tell you about
> syntax errors in your numeric expressions.
>
> There there won't be separate code paths for "parse and evaluate" and
> "skip" when handling conditionals; instead there will be just have "parse",
> with "evaluate" as a separate (bypassable) step. That improves reliability
> and simplifies maintenance. And caching of the parse tree could improve
> performance, if that matters.
>
> Backwards compatibility mode would attempt to parse expressions but also
> keep the literal text, so that when it later turns out that the variable is
> an assoc array, it can use that rather than the expression tree. This would
> of course suppress reporting expression syntax errors using bash -n.
>
> -Martin
>
> On Mon, 5 Sep 2022, 05:49 Yair Lenga, <yair.lenga@gmail.com> wrote:
>
>> Putting aside the effort to implement, it might be important to think on
>> how the h-data structure will be used by users. For me, the common use
>> case
>> will be to implement a simple, small "record" like structure to make it
>> easier to write readable code. Bash will never be able to compete with
>> Python/Node for large scale jobs, or for performance critical services,
>> etc. However, there are many devops/cloud tasks where bash + cloud CLI
>> (aws/google/azure) could be a good solution, eliminating the need to build
>> "hybrids". In that context, being able to consume, process and produce
>> data
>> structures relevant to those tasks can be useful. Clearly, JSON and YAML
>> are the most relevant formats.
>>
>> As a theoretical exercise, looking for feedback for the following,
>> assuming
>> that implementation can be done. Suggesting the following:
>> * ${var.k1.k2.k3}  -> value           # Should lookup an item via h-data,
>> supporting the regular modifiers ('-', for default values, '+' for
>> alternate, ...)
>> * var[k1.k2.k3]=value                   # Set a specific key, replacing
>> sub-documents, if any - e.g. removing any var[.k1.k2.k3.*]
>> * var[k1.k2.k3]=(h-value)              # set a specific key to a new
>> h-value
>> * ${var.k1.k2.k3.*}   -> h->value   # extract h-value string that
>> represent
>> the sub-document k1.k2.k3
>>
>> The 'h-value' representation may be the same format that is currently used
>> by the associative array. No need to reinvent here.
>>
>> Assuming the above are implemented, the missing pieces are "converters" to
>> common formats: json, yaml, and possibly XML (yet, there is still a lot of
>> those). In theory, following the 'printf' styles:
>> * printjson [-v var] h-value
>> * readjson var                         # or event var.k1
>> * printyaml [-v var] h-value
>> * readyaml var                         # or event var.k1
>>
>> To summarize:
>> * Using '.' to identify the hierarchy of the h-data - extension to bash
>> syntax.
>> * Allow setting a "node" to new value, or new sub-document - may be
>> extension
>> * Converters to/from standard formats - can be extensions
>>
>> Looking for feedback
>> Yair
>>
>