[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Unicode support in io Forge package

From: PhilipNienhuis
Subject: Re: Unicode support in io Forge package
Date: Sat, 19 Oct 2019 07:51:34 -0500 (CDT)

apjanke-floss wrote
> Hi, Octave and io maintainers,
> I'm confused by the Unicode support in the io package. In particular, 
> the functions unicode2utf8 and utf82unicode, and the "encode_utf" 
> options in some of the ods/xls read/write functions.
> What is the encoding that utf82unicode/unicode2utf8 are calling 
> "unicode" here? It looks like it's doing a single-byte encoding, 
> treating each byte as an unsigned int 0-255, and treating those 0-255 
> values directly as Unicode code point values. That's not any of the 
> standard Unicode encodings. (But I think it is exactly the same as 
> Latin-1/ISO 8859-1.)
> As I understand it, since about Octave 4.4, Octave's internal encoding 
> (that is, how it interprets Octave char arrays) is either UTF-8 or an 
> opaque array of bytes; it's never in the "system code page" or some 
> other locale-specific encoding.
> Is this UTF-8 support in io still relevant/correct? Maybe it should be 
> deprecated or renamed/removed? Since Octave now supports UTF-8, I think 
> you'd want to just leave UTF-8 text as is in all cases.

AFAIR to apply unicode2utf8 and utf82unicode there needs to be an option set
I also lost why it was included (and no time to dive in the mercurial logs
now) but there sure was a good reason for it, like bug reports etc.

In core Octave there's native2unicode and unicode2native, maybe those are a
better alternatives.


Sent from: https://octave.1599824.n4.nabble.com/Octave-Maintainers-f1638794.html

reply via email to

[Prev in Thread] Current Thread [Next in Thread]