Re: [Chicken-users] UTF-8 support in eggs

[Oleg Kolosov]

On 07/08/14 16:40, Mario Domenech Goulart wrote:
> Hi,
> 
> On Tue, 08 Jul 2014 08:57:43 +0400 Yaroslav Tsarko <address@hidden> wrote:
> 
>> Why don`t just add (use utf8) line explicitly to all the eggs that
>> handle strings? That will ultimately fix the problem and will clearly
>> indicate that egg performs string manipulations and is capable of
>> handling UTF-8 encoding.  Or am I missing something?
> 
> That'd be an option.  In my opinion, it would probably make things
> slightly better, since ASCII characters can be represented by UTF-8 in a
> single byte.
> 
> On the other hand, we risk breaking eggs that operate on latin1 text.
> UTF-8 support may also affect performance-sensitive code.
> 
> Best wishes.
> Mario
> 

IMO just enable utf8 by default and let them break. Is it's not 80's
anymore, latin1 only software should die.

We did few tests some time ago and they showed that tackling this from
Scheme side does not make worthy difference. Using pure C is much
better. Perhaps utf8 egg could enjoy some yet to be written (or found in
third party libraries) low level support from the core, so we can have
the best of the both worlds.

For specific use cases (backward compatibility, logging or minimizing
runtime size for example) it should be possible to disable multi-byte
encoding handling (i.e. current behaviour) by using compilation option
or a feature declaration.

We have a demanding application and embedded environment with not quite
fast MIPS CPU, I can help with testing and profiling things if you are
worried about performance problems.

-- 
Regards, Oleg