Re: [PATCH] add language/wisp to Guile?

[Maxime Devos]

On 04-02-2023 22:35, Dr. Arne Babenhauserheide wrote:
> Maxime Devos <maximedevos@telenet.be> writes:
>
> > > This needs an addition to the extensions via guile -x .w — I wrote
> > > that
> > > in the documentation. I didn’t want to do that unconditionally, because
> > > detecting a wisp file as scheme import would cause errors.
> >
> > If done carefully, I don't think this situations would happen.
> > More precisely:
> >
> >    * .w would be in the file extensions list.
> >
> >    * Instead of a list, it would actually be a map from extensions to
> >      languages:
> >
> >        .scm -> scheme
> >        .w -> wisp
> >
> >      With this change, (use-modules (foo)) will load 'foo.scm' as Scheme
> >      and 'foo.w' as Wisp.  (Assuming that foo.go is out-of-date or
> >      doesn't exist.)
> >
> >      (For backwards compatibility, I think %load-extensions needs to
> >      remain a list of strings, but a %extension-language variable could
> >      be defined.)
> >
> >    * "guile --language=whatever foo" loads foo as whatever, regardless
> >      of the extension of 'foo' (if a specific language is requested,
> >      then the user knows best).
> >
> >    * "guile foo" without --language will look up the extension of foo in
> >      the extension map. If an entry exists, it would use the
> >      corresponding language.  If no entry exists, it would use
> >      a default language (scheme).
>
> This sounds good, though a bit more complex than I think it should be.
>
> I think this should stick to only load Scheme if no language is detected
> to keep Scheme the default language for Guile

To my knowledge, this is the case in my proposal.  Detecting the 
language is done via the file extension, and if no known mapping exists, 
it defaults to Scheme.

> — and also to avoid
> stumbling over files that just take that extension.

While I suppose it is theoretically possible someone will write a .w 
file that contains Scheme code instead of Wisp, I'm not convinced by 
this argument.  It sounds very unlikely, and also a 'don't do that, 
then' situation.

> Checking more files
> could slow down startup and I think having multiple languages fully
> equal would risk splintering the development community.
>
> Guile is first and foremost Scheme and fast startup time is essential.
>
> More complicated is what should be done if a *.go file is detected
> during import. There I could see Guile check if a file with any
> supported extension is up to date.

Maybe the .go could contain some information on what the corresponding 
source code file name is, and Guile could read the .go without checking 
first checking for up-to-dateness. (But only reading; not loading yet!)

(There is already debugging information with such information, but to my
understanding that's for individual procedures, not the .go as a whole, 
and by using stuff like 'include' or macros there can be multiple source 
files.)

Once read, it should be easy to look up the source code file name from 
the .go and then verify whether the .go is up to date, and proceed with 
actually loading the .go (as in, put stuff in the module system, run 
top-level code, ...).

IIUC, that would be fully backwards compatible, and not cause any 
non-negligible slowdowns.

I also have an alternative proposal, more complicated and 
backwards-incompatible -- I wouldn't recommend it, but for completeness:

 * when doing (use-module (foo)) and foo.go exists in the
   $GUILE_LOAD_COMPILED_PATH, load it, and don't
   bother checking whether foo.scm, foo.w or foo.whatever exists
   or is up-to-date.  (If not done already in Guile.)

That should solve the 'I don't care what language the library Foo is 
written in, and my library Bar isn't written in Wisp so it seems 
unreasonable to have to add -x w.’, and would also avoid the need for a 
'extension -> language map' thing.  It should also be a little faster 
than what we had before.

That's for "make install", "apt-get install", "guix install" ...-like 
uses of compiled .go -- let's call them 'installed .go'.  It won't work 
for ~/.cache/guile/ccache/3.0-LE-8-4 (‘cached .go’) as for that it's 
actually important to check up-to-dateness because, well, cache.

Additionally, to support compiling software that is already installed, 
there needs to be an option to treat certain modules with the 'cache' 
behaviour even if not in the actual ~/.cache, maybe with some 
'--local-module=(stuff ...)' option (name pending).

This would be backwards-incompatible, but it could be done.

> > > readable uses
> >
> > This sentence appears to be incomplete; I might have misinterpreted it
> > below (I don't know what you mean with 'readable' -- its an adjective
> > and you are using it as a noun?).
>
> readable is a noun, yes: the readable lisp project.

Looks like you meant this: <https://readable.sourceforge.io/>.
> > > Can I write it into a string and then read it back?
> >
> > No.  [...]
>
> This sounds like I cannot go that way, because there’s a necessary
> pre-processing step in wisp-read via (match-charlist-to-repr peeked):
> [...] > This actually needs to be able to write the replacement symbols back
> into the port.
>
> > ..., for which I proposed a replacement, so do you still need to turn
> > it in a string & back?
>
> Sadly yes. Otherwise the normal reader will play tricks on the code,
> because it does not know where a symbol needs to be interpreted
> differently (i.e. where ` needs to be treated as `() even though that’s
> not in the string).

OK, too bad.  Looks like the REPR-... stuff is to stay for now.
> I meant this:
>
> (define (foo)
>    #((bar . baz))
>    #f)
> (procedure-properties foo)
> => ((name . foo) (bar . baz))
>
> I use that for doctests: [...]

I didn't know that these literal vectors get turned into procedure 
properties ... looks interesting.

Also, about wisp-unescape-underscore-and-colon and the 'only unescapes 
up to 12 leading underscores at line start (\____________)' limitation: 
I have found a solution: you can use a combination of string-every, 
symbol->string, string->symbol, substring and string-ref:

(cond ((list? code) (map wisp-... code))
      ((eq? code '\:) ':)
      ;; Look for symbols like \____ and remove the \.
      ((symbol? code)
       (let ((as-string (symbol->string code)))
         (if (and (>= (string-length as-string) 2) ; at least a single 
underscore
                  (char=? (string-ref as-string 0) #\\)
                  (string-every #\_ (substring as-string 1)))
             (string->symbol (substring as-string 1))
             code)))
     (#true code))

Greetings,
Maxime.

OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key

OpenPGP_signature
Description: OpenPGP digital signature