Re: Parse inline scheme using per-expression port (issue 557330043 by address@hidden)

[David Kastrup]

address@hidden writes:

> On 2020/02/07 15:14:52, dak wrote:
>> Ok, that should probably have been stressed somewhat stronger before: have 
>> you
>> checked the preexisting Guilev2 work in branches that Harm pointed you to?
>
> No, I didn't
>
>> Date:   Sun Sep 21 18:40:06 2014 +0200
>
> Is there more 6 year old code that is still relevant but not integrated
> to mainline?

Yes.  That's what the branches are about.

>>     Source_file::init_port: Keep GUILEv2 from redecoding string input
>> 
>> diff --git a/lily/source-file.cc b/lily/source-file.cc
>> index 5a94927a7f..5ad9c4c6e8 100644
>> --- a/lily/source-file.cc
>> +++ b/lily/source-file.cc
>> @@ -152,7 +152,11 @@ Source_file::init_port ()
>>    // we do our own utf8 encoding and verification in the parser, so we
>>    // use the no-conversion equivalent of latin1
>>    SCM str = scm_from_latin1_string (c_str ());
>> -  str_port_ = scm_mkstrport (SCM_INUM0, str, SCM_OPN | SCM_RDNG, 
>> __FUNCTION__);
>> +  scm_dynwind_begin ((scm_t_dynwind_flags)0);
>> +  // Why doesn't scm_set_port_encoding_x work here?
>> +  scm_dynwind_fluid (ly_lily_module_constant ("%default-port-encoding"),
>> SCM_BOOL_F);
>> +  str_port_ = scm_open_input_string (str);
>> +  scm_dynwind_end ();
>>    scm_set_port_filename_x (str_port_, ly_string2scm (name_));
>>  }
>> 
>> Now it is a valid question why this isn't, GUILEV2-guarded, already in the 
>> main
>> code base.  Maybe we should integrate all of the already existing Guilev2 
>> work
>> into master with priority in order to avoid duplicate work.
>
> Yes, that would be great idea. Is there any other pending work that you
> know of?
>
> I think the patch you quote here is doing the wrong thing, though. If
> you have Scheme code with UTF-8 encoded data embedded into it, it will
> get parsed out as Latin1.

Oh, definitely.  Just like Guile-1.8 does.  This is a "Get things to
work like previously" patch that keeps abominations like
ly:wide-char->utf-8 operative as well as required like before.  All
strings taken from the input are essentially byte strings and utf-8 has
to be handled at the byte level.

It is not a desirable end point for Guilev2 migration but a migration
strategy designed for getting back something working as before.  Which
is a starting point from which one can then explore the task of getting
to an environment that looks like Unicode rather than bytes from the
Scheme side.

-- 
David Kastrup