Re: Possible solution for special characters in makefile paths

[Frank Heckenbach]

Paul Smith wrote:

> > An alternative, of course, would be a multi-character encoding, with
> > the advantage that all characters can be encoded uniquely (same
> > principle as UTF-8).
> 
> I'm not sure I see.  The reason I suggested those characters for the
> mapping characters is that most variable-length locale encodings
> preserve the ASCII values for the first 127 characters, and there are
> values in that range which are not used, or hardly ever used (for
> example, not part of the Portable Character Set defined by POSIX), in a
> text file such as a makefile or in pathnames.
> 
> Once you get beyond the first 127 characters it seems to me that the
> similarities drop off precipitously.

No, sorry, I didn't mean actually using the UTF-8 encoding, just the
same principle, using lower characters. Indeed, I realize, it's
still simpler than UTF-8 since we don't need variable-length
encodings. E.g. using a pair of characters out of a set of 16 (e.g.
1..6 and 16..25, all very rarely used AFAIK) one could encode any
byte value (including these characters themselves), i.e. allow any
character (in any charset), at the expense of strings getting up to
twice as long.

> > > 1. Any makefile that uses one of the chosen mapping characters will
> > >    fail.  We can detect this during makefile parsing and throw an
> > >    error, so this will not be a silent problem.
> > 
> > For my progress messages (cf.
> > http://lists.gnu.org/archive/html/bug-make/2013-04/msg00060.html), I
> > do indeed use \001 as an indicator (for basically the same reasons
> > you suggest to use it for encoding). It wouldn't be a major problem
> > for me to switch to some other indicator, so that's just to say that
> > based on my sample of size 1, such things do occur, portable or not.
> 
> Well, with a userbase the size of GNU make's, no change has zero impact.
> However, it is a bit troubling that we've already found a counter-case.

As I said, don't worry too much about my case. If you choose to
implement such a proposal, I can surely change my scripts in time.
But other users who don't read this list may be taken by surprise
when it comes out.

> > > 2. Any makefile using the chosen "quoting" token will break; i.e.
> > >    if some makefile today has "[ = foo" then uses "$[" later, and
> > >    we choose $[...] for quoting, this will fail.  It would have to
> > >    be changed to use "$([)" instead.  Same for ` if we choose
> > >    $`...` etc.
> > 
> > FWIW, I grepped my Makefiles for "$[" and the only occurrences I
> > found were bash arithmetic expansions like this:
> 
> What about $` ?

Doesn't occur in my Makefiles.

> > SHELL=/bin/bash
> > foo:
> >     n=3; echo $$[$$n + 1]
> > 
> > I suppose that's harmless, since the "$" here is escaped and
> > wouldn't be parsed as part of "$[". Still, it would be worthwhile to
> > add a test of this kind if you implement this ...
> 
> Yes, this is harmless.  I'm not sure what you mean by "add a test of
> this kind"...?  You mean, add something to the regression test suite?
> Or add a test while reading makefiles?
> The way we'd test this would be to watch variable assignments and check
> if the variable name was the new quoting token.

I meant the regression test suite. You could just add a test like
the above fragment. If relying on a bash feature is not appropriate,
you could do something like echo '$$[foo]' and check that it
produces correct output.

> Whether we do this or not depends, to me, on whether we want to allow
> users to use that variable name with the caveat that they must use
> parentheses or braces around any references.  For example, this could be
> considered OK because it's not ambiguous:
> 
>     [ = foo
>     X := ${[}
> 
> While this would not be correct (probably give an error about malformed
> quoting sequence or something):
> 
>     [ = foo
>     X := $[
> 
> If we want to allow this form, then I wouldn't want to warn on the "[ =
> foo" line, at least by default.  Maybe if a warnings flag were enabled.

I don't have a strong opinion here, since special characters in
variable names feel strange to me and I never use them. Allowing the
former would be the most backward-compatible, sure. While we're at
it, what about spaces in variable names, should they be un-/escaped
automatically, i.e. should all of the following work? (Again, I
don't really care myself, but it's worth thinking about.)

n = a b
$n = x
m = c d
$[c d] = y

foo:
        echo ${${n}}
        echo ${a b}
        echo ${$[a b]}
        echo ${${m}}
        echo ${c d}
        echo ${$[c d]}

PS: While experimenting with the above, it seems I found a bug.
With a Makefile consisting solely of:

$[x]

make says:

Makefile:1: *** missing separator (did you mean TAB instead of 8 spaces?).  
Stop.

Looking at the code, I don't readily understand the bug (but eval()
is a complex function and I may be missing something). If you can't
reproduce it, I can probably do some more debugging.