Re: Escaping quotes in docstrings, Was: A simple solution to "Upcoming loss of usability ..."

[Paul Eggert]

Dmitry Gutov wrote:
> Why not put `escaped' (or rename that `translated', maybe) on those generated
> regions as well?

Yes, in theory it could be done.  But see below.

> > Second, and more generally, the overall approach seems error prone.  It
> > asks programmers to mark every quote character that should not be
> > transformed.  But characters can come from many different sources, and
> > it's hard to keep track of each place that can insert user data of some
> > sort.
>
> So far I've only seen a few places of complication nearby: key translations,
> keymap translations, and the escaping mechanism itself.

I'm afraid this is because you've only started looking.  Even if we limit 
ourselves to the regions you mention, the relevant code is scattered all over 
the place and it's hard to find exactly where to put 'escaped' or 'translated' 
into those regions.  For example, one place might be the following code in 
keymap.c's single_key_description function:

          Lisp_Object result;
          char *buffer = SAFE_ALLOCA (sizeof "<>"
                                      + SBYTES (SYMBOL_NAME (key)));
          esprintf (buffer, "<%s>", SDATA (SYMBOL_NAME (key)));
          result = build_string (buffer);
          SAFE_FREE ();
          return result;

Since KEY can be an arbitrary symbol, quite possibly its characters should be 
marked as 'escaped' or 'translated', which would mean adding this:

  put_text_property (1, 1 + SCHARS (SYMBOL_NAME (key)), Qescaped, Qt, result);

before the return.  Unfortunately it's not at all obvious whether this is needed 
unless one examines all the ways that C-h can put text into the *Help* buffer 
(which I haven't done).  And one must do this potentially every time a string or 
symbol name is looked at.  Plus, there are a lot of possibilities for off-by-one 
errors: for example, that '1 +' might easily be forgotten.  Plus, even then 
there are opportunities for bugs: the above call to 'put_text_property' is not 
correct if the symbol name contains a NUL byte.  Plus, there are other areas 
that will require this sort of complication, e.g., button labels, *Apropos* buffers.

In contrast, it's much easier to look through the code for ` inside a string, 
and mark the exceptional characters that are intended to be quotes and that are 
not automatically translated already.

> (we don't know all the possible sources the characters can
> come from? that's not very good).

No, actually, it's a good thing.  It's nice that programs can put arbitrary text 
into *Help* buffers, and that that we don't need to worry about where the text 
came from: it just works.  That's a simple interface, and simple interfaces are 
good.

> By the way, when doing translation in Lisp, instead of using help-mode-finish,
> we can instead examine all the functions that generate help buffers for Lisp
> stuff, and translate only the docstrings, when they are being inserted.

That's what the master branch is doing now, no?