Re: Encoding of LilyPond console output

[David Kastrup]

David Kastrup <address@hidden> writes:

> I am indeed top-posting.

<URL:http://code.google.com/p/lilypond/issues/detail?id=2173>

> Wilbert Berendsen <address@hidden> writes:
>
>> Everything boils down to the question in which encoding the 8bit
>> console output of LilyPond is presented. I was always assuming that
>> this was UTF-8, which worked correctly on Linux. I.e. both
>> filenameswith accented letters and translated messages (such as
>> French, with many accented letters) always showed up correctly in the
>> LilyPond console output. Which is read by Frescobaldi as an 8bit
>> bytestream and decoded into unicode strings using the UTF-8 encoding.
>>
>> Then, on Windows, I discovered that filenames do not use UTF-8
>> encoding, but rather 'mbcs' or something like that (what is returned
>> by sys.getfilesystemencoding() in Python).
>>
>> So I changed Frescobaldi to use that encoding when reading LilyPond
>> console output, but then we discovered that translated messages (such
>> as the French ones) with accented characters do show in a garbled
>> encoding (clearly showing something like UTF-8 displayed as Latin1).
>>
>> So again I changed Frescobaldi, and now it reads the console output
>> byte stream and parses that for file references (such as:
>> file.ly:12:3: error: blabla) and decodes those filenames using the
>> filesystem encoding, and the rest using UTF-8.
>>
>> This seems to work well: file references are correctly parsed and
>> messages are readable still.
>
> It will likely cease working at some point of time.  Lilypond is
> supposed to be UTF-8.
>
>> So everything thrown together: is my analysis of the mixed output
>> encodings LilyPond uses on stdout and stderr correct?
>
> Probably.
>
>> And in line with this: can LilyPond be made more aware of this, and
>> use the same encoding for all output (correctly encoding filenames)?
>
> It will have to at one point of time.  I am currently preparing a patch
> where it will reject all non-UTF-8 input, and of course this includes
> filenames in \include and similar statements.  I have no idea how to
> deal with the command line on Windows.

-- 
David Kastrup