Re: guile-devel Digest, Vol 260, Issue 25

[Marc Nieper-Wißkirchen]

I would like to comment on what I think are common misconceptions about the RnRS library system.

1. The RnRS library system is neither a prerequisite for being able to write portable code nor is it particularly helpful in this regard. The RnRS library system should better be called a module system, as its primary function is to create clean (top-level) lexical environments. The ability to do so can help to write portable code, but it can also be a hindrance to doing so. For example, the sample implementation of SRFI 9 is an R5RS implementation that works by redefining some vector primitives. With the module system of R6RS and R7RS, such an implementation strategy is no longer possible.

2. To understand what's needed to easily write useful portable code, we have to focus our attention away from the red herring called library system. What is needed is, firstly, a broad-ranging standard (e.g. without the necessary standardized primitives, you can never write a portable HTTP library). Secondly, implementations need to adhere to the standard as faithfully as possible. Thirdly, there needs to be an underlying model from which one can draw efficiency expectations (e.g., as long as implementations do not agree on implementing call/cc efficiently, useful portable code cannot be centred around algorithms employing call/cc). 

These conditions are in place for a programming language like C. While the C standard itself is not that broad, implementers agree on an ABI for each platform so that platform features can be added portably. C implementations like GCC or Clang work hard to fix deviations from the standard. Lastly, implementations typically agree on an implementation model so that the programmer knows what kind of code is efficient and what kind of code is not.

The Scheme world is different. Firstly, the standard is fairly minimal. SRFIs try to broaden the standard, but they are not universally implemented and accepted. Contrary to the standards, SRFIs ultimately only reflect the opinion of their respective authors. Secondly, many implementations, including many big implementations (like Guile), deviate from the standard when it makes sense for them (e.g., Guile implements, first and foremost, the Guile programming language and offers an implementation of RnRS on top but would never compromise the further development of the Guile language over pedantic adherence to RnRS). Thirdly, many Scheme implementations have been created for interpreter/compiler research so that different implementation models with different efficiency expectations come naturally to the Scheme language.

3. The Scheme standards RnRS make no assumptions about the representations of libraries on the filesystem. Not only is the encoding of the filename undefined, but it is even unspecified whether a library has to reside on the filesystem and whether libraries are loaded automatically (versus an explicit load command to make the library definition known to the implementation). This makes discussions about the encoding of, say, ":" moot when it comes to portability. As far as the standard is concerned, installing a library is, in general, more than just copying files. Package managers like Akku or Snow can be used for this; they know about the target implementation and can rename or move files as needed (moving is generally necessary even for the R7RS, including syntax, which was mentioned because R7RS does not prescribe where to find the file). This issue is orthogonal to portability but no less important to make it convenient for the user. Here, we have similarities to the C standard, which also does not prescribe the system interface to the compiler.

4. The library names of the form "(srfi :1)" are non-standard but a convention suggested in SRFI 97. Even if they are supported, only a part of the SRFI effort is reflected because many features proposed by SRFIs are not in the form of libraries. Allowing numbers in library names makes certain syntactic extensions (as some found in Chez Scheme) impossible. In the syntax-case model of R6RS (which is also the basis of Guile's expander), only identifiers like ":1" carry lexical information (a set of marks and substitutions in the R6RS model), numbers and other Scheme datums do not. The authors of R7RS were presumably unaware of it. A unifying approach would, therefore, be to use the ":1" version and to offer the number version "1" solely for backwards compatibility with R7RS. As said above, this has nothing to do with the encoding of the library name as a path on the filesystem. An implementation can still look up "(srfi :1)" under the path "srfi/1.sls" if it wishes to do so. Library definitions are self-delimiting forms, so more than one library definition can be included in a file.

Let me also add a personal remark about the "ice-9" namespace: I do not see a particular problem with it. All that is needed is a well-known name (well-known for those who have browsed which libraries are available) that is sufficiently unique. Introducing a new name can be counterproductive as it reduces the name recognition of the old name (which will still be around). Moreover, it introduces friction: programmers who study existing code will have to know both the old name and the new alias, making things more complicated in the end. Unless there is a technical benefit to doing so, I advise against replacing "ice-9" with "guile". Another reason for my wariness is that not everything in the ice-9 namespace is Guile-specific in the strict sense. For example, (ice-9 regex) seems to be a library that can live outside of Guile's universe as well. Finally, talking about "ice-9" (or even about names like "define-module2") feels like bikeshedding.

Cheers,

Marc

Am Sa., 20. Juli 2024 um 22:35 Uhr schrieb <guile-devel-request@gnu.org>:

Send guile-devel mailing list submissions to
        guile-devel@gnu.org

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.gnu.org/mailman/listinfo/guile-devel
or, via email, send a message with subject or body 'help' to
        guile-devel-request@gnu.org

You can reach the person managing the list at
        guile-devel-owner@gnu.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of guile-devel digest..."


Today's Topics:

   1. Portable imports (Lassi Kortela)
   2. RE: Portable imports (Maxime Devos)
   3. RE: Portable imports (Maxime Devos)
   4. Encoding library names (Lassi Kortela)
   5. RE: Encoding library names (Maxime Devos)


----------------------------------------------------------------------

Message: 1
Date: Sat, 20 Jul 2024 21:42:37 +0300
From: Lassi Kortela <lassi@lassi.io>
To: Maxime Devos <maximedevos@telenet.be>, "Dr. Arne Babenhauserheide"
        <arne_bab@web.de>
Cc: Attila Lendvai <attila@lendvai.name>, Greg Troxel
        <gdt@lexort.com>, MSavoritias <email@msavoritias.me>,
        "guile-devel@gnu.org" <guile-devel@gnu.org>
Subject: Portable imports
Message-ID: <04293f0f-3a83-4ebc-8413-1a936caaeb57@lassi.io" target="_blank">04293f0f-3a83-4ebc-8413-1a936caaeb57@lassi.io>
Content-Type: text/plain; charset=UTF-8; format=flowed

> I don’t know if ‘(import ...)’ is standard either (sure it is as part of
> ‘define-library’, but I didn’t find it on its own in r7rs.pdf),

(import ...) is standard in both R6RS and R7RS, and supported by every
serious implementation of those standards. Please spread it.

R7RS talks about "programs" and "libraries". These are technical terms
with precise meanings.

A "program" corresponds to your typical Scheme script. IIRC it _has_ to
start with (import ...).

A "library" is a (library ...) [in R6RS] or a (define-library ...) [in
R7RS]. You can type (import ...) inside either.

>  > https://srfi.schemers.org/srfi-97/srfi-97.html:
>
>  >A SRFI Library can be referenced by number, as in
>
>  >(srfi :1),
>
> (srfi 1) is problematic, since ‘1’ is not an symbol (#{1}# is, but
> that’s not what has been choosen in SRFI 97).

In R7RS non-negative integers can be library name parts. Since these
library names look natural, it would be good to backport this to R6RS
implementations.

The colon causes endless grief when mapping library names to file names.
For example, look at all the %3a in
https://github.com/arcfide/chez-srfi. That's not even the worst of it.



------------------------------

Message: 2
Date: Sat, 20 Jul 2024 21:18:53 +0200
From: Maxime Devos <maximedevos@telenet.be>
To: Lassi Kortela <lassi@lassi.io>,  "Dr. Arne Babenhauserheide"
        <arne_bab@web.de>
Cc: Attila Lendvai <attila@lendvai.name>, Greg Troxel
        <gdt@lexort.com>,  MSavoritias <email@msavoritias.me>,
        "guile-devel@gnu.org" <guile-devel@gnu.org>
Subject: RE: Portable imports
Message-ID:
        <20240720211840.pvJf2C00709gYMG06vJgRX@michel.telenet-ops.be" target="_blank">20240720211840.pvJf2C00709gYMG06vJgRX@michel.telenet-ops.be>
Content-Type: text/plain; charset="utf-8"

>In R7RS non-negative integers can be library name parts. Since these
library names look natural, it would be good to backport this to R6RS
implementations.

Then (library [...] (import (srfi 1)) [...]) would work, and since ‘library’ is (R6RS) standard and reasonably portable it would then appear that (srfi 1) is (R6RS) standard and portable, whereas it isn’t R6RS, and hence not a good idea to backport.

>The colon causes endless grief when mapping library names to file names.
>For example, look at all the %3a in
https://github.com/arcfide/chez-srfi. That's not even the worst of it.

I don’t think this is a problem for Guile? I don’t recall to what extent, but (srfi ...) modules are somewhat special-cased in Guile (or maybe it was integers in general for define-library) – maybe its implementation of define-library translates (srfi 1) to (srfi srfi-1) (I don’t recall the specifics). Hence, the file can simply be named “srfi/srfi-1.scm”.

For compatibility, both(**) (srfi srfi-N) (← non-standard Guile thingie) and (srfi :N) need to be supported anyway for ‘define-module’ (← Guile-specific interface), so which of them determines the file name is just a matter of convenience.

Also, AFAIK that %3a encoding isn’t necessary (and neither recognised(^)) in Guile – I don’t think Guile does any percent encoding(*). I think naming the file “srfi/:1.scm” would work fine, although it’s not something I’ve tried before. (There might be a problem with Makefile if ‘make’ doesn’t like the :, but I have some ideas for simple ways around that.)

(*) implication: you can’t have two different modules (foo/bar) and (foo bar) in Guile.
(^) (srfi %3a1) would mean the module has literally (srfi %3a1) as name.
(**) not entirely true, only supporting (srfi srfi-N) (in define-module) would be compatible, but that does not seem to be the future.

Best regards,
Maxime Devos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnu.org/archive/html/guile-devel/attachments/20240720/deaf1eb8/attachment.htm>

------------------------------

Message: 3
Date: Sat, 20 Jul 2024 21:23:53 +0200
From: Maxime Devos <maximedevos@telenet.be>
To: Lassi Kortela <lassi@lassi.io>,  "Dr. Arne Babenhauserheide"
        <arne_bab@web.de>
Cc: Attila Lendvai <attila@lendvai.name>, Greg Troxel
        <gdt@lexort.com>,  MSavoritias <email@msavoritias.me>,
        "guile-devel@gnu.org" <guile-devel@gnu.org>
Subject: RE: Portable imports
Message-ID:
        <20240720212340.pvPf2C00709gYMG01vPfWY@laurent.telenet-ops.be" target="_blank">20240720212340.pvPf2C00709gYMG01vPfWY@laurent.telenet-ops.be>
Content-Type: text/plain; charset="utf-8"

>R7RS talks about "programs" and "libraries". These are technical terms
with precise meanings.
>A "program" corresponds to your typical Scheme script. IIRC it _has_ to
start with (import ...).

I’ve found it in r7rs.pdf now:

> 7.1.6. Programs and definitions
> <program> → <import declaration>+ <command or definition>+
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnu.org/archive/html/guile-devel/attachments/20240720/367b46f7/attachment.htm>

------------------------------

Message: 4
Date: Sat, 20 Jul 2024 22:46:27 +0300
From: Lassi Kortela <lassi@lassi.io>
To: Maxime Devos <maximedevos@telenet.be>, "Dr. Arne Babenhauserheide"
        <arne_bab@web.de>
Cc: Attila Lendvai <attila@lendvai.name>, Greg Troxel
        <gdt@lexort.com>, MSavoritias <email@msavoritias.me>,
        "guile-devel@gnu.org" <guile-devel@gnu.org>
Subject: Encoding library names
Message-ID: <80a09f9e-a84e-47ef-9fbd-72caf22fe948@lassi.io" target="_blank">80a09f9e-a84e-47ef-9fbd-72caf22fe948@lassi.io>
Content-Type: text/plain; charset=UTF-8; format=flowed

>  > In R7RS non-negative integers can be library name parts. Since these
> library names look natural, it would be good to backport this to R6RS
> implementations.
>
> Then (library [...] (import (srfi 1)) [...]) would work, and since
> ‘library’ is (R6RS) standard and reasonably portable it would then
> appear that (srfi 1) is (R6RS) standard and portable, whereas it isn’t
> R6RS, and hence not a good idea to backport.

For the time being, (library ...) is only available in R6RS
implementations. But the next report (tentatively titled R7RS-large) is
on track to be a merger of R6RS and R7RS, and hence will most likely
support both (library ...) and (define-library ...) while merging their
semantics in some way.

I would agree that interop between strict R6RS and other dialects of
Scheme is important. To that end, the option to use numbers in R6RS
library names using the : prefix is good to have. (R6RS does not have
the vertical bar notation |123| to turn numbers into symbols, so strict
R6RS code cannot even rely on that notation to encode numerical library
name parts).

A further complication is that :123 is a keyword in some Scheme
implementations. (This syntax comes from Common Lisp and Emacs Lisp,
perhaps going as far back as Maclisp.) It might be best if any leading
colon in a library name part is simply removed.

> > The colon causes endless grief when mapping library names to file names.
> > For example, look at all the %3a in
> > https://github.com/arcfide/chez-srfi. That's not even the worst of it.
>
> I don’t think this is a problem for Guile? I don’t recall to what
> extent, but (srfi ...) modules are somewhat special-cased in Guile (or
> maybe it was integers in general for define-library) – maybe its
> implementation of define-library translates (srfi 1) to (srfi srfi-1) (I
> don’t recall the specifics). Hence, the file can simply be named
> “srfi/srfi-1.scm”.
>
> For compatibility, both(**) (srfi srfi-N) (ß non-standard Guile thingie)
> and (srfi :N) need to be supported anyway for ‘define-module’ (ß
> Guile-specific interface), so which of them determines the file name is
> just a matter of convenience.
>
> Also, AFAIK that %3a encoding isn’t necessary (and neither
> recognised(^)) in Guile – I don’t think Guile does any percent
> encoding(*). I think naming the file “srfi/:1.scm” would work fine,
> although it’s not something I’ve tried before. (There might be a problem
> with Makefile if ‘make’ doesn’t like the :, but I have some ideas for
> simple ways around that.)

Guile should support non-negative integers in any library name, not only
for the SRFI libraries. R7RS allows them anywhere.

I would recommend implementing the %-encoding of arbitrary UTF-8 bytes
(several Scheme implementations do it when translating library names to
file names) but to avoid using it in practice.

The : prefixes should be stripped before encoding a library name as a
file name.

FAT, NTFS (and probably other file systems) do not allow colons in file
names. I am not aware of any file system that forbids the percent sign.

> (*) implication: you can’t have two different modules (foo/bar) and (foo
> bar) in Guile.

Percent-encoding the slash solves this easily.

> (^) (srfi %3a1) would mean the module has literally (srfi %3a1) as name.

AFAIK that is allowed by RnRS.



------------------------------

Message: 5
Date: Sat, 20 Jul 2024 22:35:35 +0200
From: Maxime Devos <maximedevos@telenet.be>
To: Lassi Kortela <lassi@lassi.io>,  "Dr. Arne Babenhauserheide"
        <arne_bab@web.de>
Cc: Attila Lendvai <attila@lendvai.name>, Greg Troxel
        <gdt@lexort.com>,  MSavoritias <email@msavoritias.me>,
        "guile-devel@gnu.org" <guile-devel@gnu.org>
Subject: RE: Encoding library names
Message-ID:
        <20240720223521.pwbM2C00209gYMG01wbMcg@andre.telenet-ops.be" target="_blank">20240720223521.pwbM2C00209gYMG01wbMcg@andre.telenet-ops.be>
Content-Type: text/plain; charset="utf-8"

>>  > In R7RS non-negative integers can be library name parts. Since these
>> library names look natural, it would be good to backport this to R6RS
>> implementations.
>>
>> Then (library [...] (import (srfi 1)) [...]) would work, and since
>> ‘library’ is (R6RS) standard and reasonably portable it would then
>> appear that (srfi 1) is (R6RS) standard and portable, whereas it isn’t
>> R6RS, and hence not a good idea to backport.

>For the time being, (library ...) is only available in R6RS
implementations. But the next report (tentatively titled R7RS-large) is
on track to be a merger of R6RS and R7RS, and hence will most likely
support both (library ...) and (define-library ...) while merging their
semantics in some way.

This is not _yet_ the case – AFAIK R7RS-large is still in progress. So, too soon to implement it for ‘library’ yet – ‘library’ is currently R6RS.

>I would agree that interop between strict R6RS and other dialects of
Scheme is important.
> To that end, the option to use numbers in R6RS
library names using the : prefix is good to have. (R6RS does not have
the vertical bar notation |123| to turn numbers into symbols, so strict
R6RS code cannot even rely on that notation to encode numerical library
name parts).

In the case of SRFI, yes, since that’s what the relevant SRFI says the module names are, but you are formulating this much more generally.

>A further complication is that :123 is a keyword in some Scheme
implementations. (This syntax comes from Common Lisp and Emacs Lisp,
perhaps going as far back as Maclisp.) It might be best if any leading
colon in a library name part is simply removed.

Guile isn’t one of those, so it’s not a problem. As I understand it, (a priori) (foo N), (foo :N) and (foo |N|) are three different module names, so this removal is simply incorrect (barring changes to RnRS). As such leading colons should be kept.

As part of a ‘module name -> file name’ mapping it seems a reasonable choice, but that’s a different matter.

(Not claiming that an implementation should in general support different (foo :N) (foo N) (foo |N|) modules, only that it should recognise them as different names.)

>> [...]
>Guile should support non-negative integers in any library name, not only
for the SRFI libraries. R7RS allows them anywhere.

I never claimed it should be restricted to SRFI libraries. My comment was about the colon and the problems it would cause (more precisely, the lack of problems) (and about not doing it for R6RS library forms).

>I would recommend implementing the %-encoding of arbitrary UTF-8 bytes
(several Scheme implementations do it when translating library names to
file names) but to avoid using it in practice.

Err, no, I recommend _not_ doing that. Unicode brought us the option to be able to just type characters without special tricks, I don’t want Guile to regress to ASCII here.

Now, if Guile were to support both %-encoding (e.g. for : on Windows situations) but also supported (and preferred) just literally including the actual character in the filename (Unicode as intended, applied to file names), that would be fine, but since Guile’s current module implementation just directly maps module names to file names (+ search path), that’s currently not an option (start-up performance implications).  (I don’t think this is an unsurmountable problem, but it does require some reorganization in how Guile libraries are packaged and how Guile searches for .go/.scm.)

(Other option: %-encode only disallowed characters so there is a unique corresponding file name (modulo search paths), but which characters are disallowed depend on file system and OS, so that’s not practical.)

>The : prefixes should be stripped before encoding a library name as a
file name.
>FAT, NTFS (and probably other file systems) do not allow colons in file
names. I am not aware of any file system that forbids the percent sign.

NTFS supports colons just fine(*), it’s Windows that places restrictions on file names.
So, some change is indeed necessary (but not necessarily just always stripping the :, other options exist as well.)

(*) not completely sure – ‘:’ can refer to alternate data streams, but I don’t know whether that’s an NTFS or a Windows thing

>> (*) implication: you can’t have two different modules (foo/bar) and (foo
>> bar) in Guile.
>
>Percent-encoding the slash solves this easily.

See above for how percent-encoding is a problem of its own.

Best regards,
Maxime Devos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gnu.org/archive/html/guile-devel/attachments/20240720/7a7f4630/attachment.htm>

------------------------------

Subject: Digest Footer

_______________________________________________
guile-devel mailing list
guile-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/guile-devel


------------------------------

End of guile-devel Digest, Vol 260, Issue 25
********************************************