Re: Unclosed quotes on heredoc mode

[Robert Elz]

    Date:        Thu, 18 Nov 2021 15:46:10 -0500
    From:        Chet Ramey <chet.ramey@case.edu>
    Message-ID:  <5c36d290-0e6e-2aa0-f388-20ec9369a6a4@case.edu>

  | Yeah, that's a bug. But it's probably baked in.

Very.  Just stopping parsing expansions while reading the here doc
delim string would be easy (well, possible anyway), but the expansion
syntax needs to be handled still, ${x- } is still one word, not two
which it would be if $ processing was simply disabled.   Further
in cmd <<$( random text 'xyz' )   the end delim most likely
includes the literal ' chars.   That would be very difficult to
deal with.  Similarly if a cmdsub included a $' string, that
should not be expanded...


  | OK, that's clearly a bug. Is this specific to the literal string `$PATH',
  | or are there more things that trigger it?

Any $ expansion strings.  Or ``.

The here doc end delim is the only place these ever appear unquoted
when they will not later be expanded.

  | The error message uses some sloppy language, but that's neither here nor 
there

I know, and I could easily fix that, but while:

  | -- this is a perfectly valid script:
  |
  | cat <<$PATH
  | hello
  | $PATH

it is, no-one ever writes things like that in real code (so no-one
ever sees the error message in practice).

I have been looking for a reasonable way to fix this for a while,
without sacrificing the advantages gained by doing things this way.
Nothing reasonable has occurred to me yet, and it is a very low
priority issue.

  | You're certainly free to consider it a bug,

I do.

  | and not to consider the compatibility concerns that inspired its inclusion 
in
  | bash in the first place,

illusory compat issues.  I have no idea what inspired this initially, but
my guess would be a code bug no-one noticed.

  | but let's not pretend that this is something that
  | died out a long time ago.

the implementations exit, the need for it does no.

  | > Further, no-one (not anyone I
  | > have ever seen) deliberately relies upon the here doc ending at EOF, not
  | > even if a here doc is in a -c command string or similar).
  |
  | You never really know, do you?

So just how many complaints do you get about the warning message?
"ksh doesn't complain wbout this, why does bash?"

I have seen zero of those on bug-bash.  My guess is that if wnyone
does ever see it they simply fix their code, there's never a need
to do otherwise.

  | Which instance of `ola"'? The first or the second?

The first.

  | This cannot be a serious question unless you mean the second.

It is a very serious question, but not as to what should hppen
but how the standard needs to describe it.

  | The delimiter is a `word', and we both know what a shell word is.

yes, but that's irrelevant, it is merely a coincidence here that
the newline in question occurs in the delimiter.
Another example
        cat <<EOF > file ; echo "abc
        def
        EOF
        ghi" \
        EOF
        EOF
What is the here doc, and what does echo say.

The first newline after the << is the one after abc.
Do remember that here doc data collection is entirely a
lexical issue, that's why tgey dot appear anywhere in
the sh grammar.

  | In other messages from both of us, we agree that
  | the delimiter is "ola\nI,\nola\nola". The here document body starts at the
  | next newline following that delimiter.

Sure, this was not the best example of the problem (with the std)

  | The newline after the delimiter is both, but sure, newline token would
  | probably work better.

The example above shows the issue better.  That includes the \newline
which can only be a \ newline because the 2nd char there is a newline,
and that has to be seen at the lexical level.

  | So it doesn't read `lines' in the POSIX sense? Huh. Who knew?

For this.  No.   An extension.  One that comes for feee.

kre