bug#32442: 26.1; reftex: TOC problems with section titles containing double quotes

[Arash Esbati]

Peter Pichler <address@hidden> writes:

> Am 15.08.2018 um 21:49 schrieb Arash Esbati:
>> Hi Peter,
>> I think this is RefTeX's subtle way to say: "Please also update your TeX
>> distro to latest TeXlive or MikTeX, change your file to this and use
>> UTF8 encoding with `C-x RET f prefer-utf-8 RET'":
>>
>>      \documentclass{article}
>>      \usepackage[ngerman]{babel}
>>
>>      \usepackage[T1]{fontenc}
>>      \usepackage{lmodern}
>>
>>      \begin{document}
>>
>>      \tableofcontents
>>
>>      \section{Könige}
>>      \label{sec:konige}
>>      Könige sind selten.
>>
>>      \end{document}
>>
>> Yes, this seems to be a bug, thanks for the report.  But I can't tell
>> why.  Will try to investigate, someday :-)

Hi Peter,

please keep address@hidden in CC, thanks.

> actually, it is the other way round ;-) Because I had to update my
> MikTeX version, I also had to update to a more recent emacs/auctex
> installation. I am aware that I could use the äöüß instead of "a
> etc. but the last time this resulted in serious problems with
> spellchecking which I could not resolve within reasonable time.

Then I suggest you give it a new try and fix this issue.

> In the meanwhile, I tried to find find out what happened. It appears
> that the problem occurs in the function reftex-context-substring
> called in line 19 of the function reftex-section-info which is called
> in turn in line 73 of the function reftex-parse-from-file. Without
> double quotes, the (up-list 1) in line 23 of the function
> reftex-parse-from-file works normally and the argument of the section
> command is extracted correctly. If the argument of the section command
> contains double quotes, the (up-list 1) results in an error and
> reftex-parse-from-file takes the 150 characters (or until eof)
> beginning with the argument of the section command. Strangely enough,
> up-list called interactively or out of an interactive function (even
> within the condition-case) causes no problems at all. Unfortunately,
> this erratic behavior exceeds my experiences with emacs-lisp by far
> and I hope, you have more luck.

You were on the right track here.  This behavior isn't erratic when you
look at the definition of `reftex-parse-from-file'.  There you see that
the actual parsing is done inside a `reftex-with-special-syntax'.  This
macro saves the current syntax table, activates the one called
`reftex-syntax-table', does the job and restores the saved one.

`reftex-syntax-table' is defined like this:

    (setq reftex-syntax-table (copy-syntax-table))
     (modify-syntax-entry ?\( "." reftex-syntax-table)
     (modify-syntax-entry ?\) "." reftex-syntax-table)

Here you see that it doesn't touch the syntax for `"' which is used to
delimit string constants.  With this setup, `K"onig' seems to `up-list'
like a string `onig' which doesn't end, hence it doesn't work.  You can
easily fix this by writing `K\"onig', i.e., escaping the quote and it
works as expected.

But then again, I recommend you let OT1-encoded fonts and \"-constructs
go and try to fix the other issue.

Here the file I tested with:

    \documentclass{article}
    \usepackage[ngerman]{babel}

    \usepackage[T1]{fontenc}
    \usepackage{lmodern}

    \makeatletter
    address@hidden
    \makeatother

    \begin{document}
    \tableofcontents

    \section{Könige}
    \label{sec:konige}
    Könige sind selten.

    \section{K"onig}
    \label{sec:konige-1}
    K"onige sind selten.

    \section{K\"onig}
    \label{sec:konige-2}
    K"onige sind selten.

    \section{K"onige\expandafter\Gobble\string"}
    \label{sec:konige-3}
    K"onige sind selten.
    \end{document}

Best, Arash