[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Racket packages / build system
From: |
Christopher Lemmer Webber |
Subject: |
Re: Racket packages / build system |
Date: |
Mon, 19 Oct 2020 13:04:03 -0400 |
User-agent: |
mu4e 1.4.12; emacs 26.3 |
Bonface M. K. writes:
> Christopher Lemmer Webber <cwebber@dustycloud.org>
> writes:
>
>> Bonface M. K. writes:
>>
>>> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>>>
>>>> Just a heads up that right now you *can* run Racket-on-Chez, but soon
>>>> Racket-on-Chez will be the "default"... maybe it's a good idea to see
>>>> how hard it will be to make a racket-on-chez package variant.
>>>>
>>>> I'm interested in looking at that but it'll probably have to be a while
>>>> before I can do so... if someone does so before me, I'll be interested
>>>> in beta testing at least.
>>>>
>>>> But also leaving this here as a self-reminder so we aren't surprised
>>>> when it becomes a more important thing to deal with :)
>>>>
>>>>
>>>
>>> Also, since we are talking about Racket, what
>>> happened to having a racket build system?
>>
>> There's interest in it, and Dimos made interesting progress towards
>> figuring out some of the key problems... there's been interest beyond
>> that but sadly it seems like organizing the energy to work on it hasn't
>> happened for whatever reason yet.
>>
>
> Can I volunteer on this task? There's some work in
> my team done in Racket; and it would be of great
> interest to us to have a working Racket build
> system. I can set apart some time to work on
> this. I'd ask for alot of guidance/ review on this
> though :)
YES! Please do. Let's talk. You can ping me on IRC also, dustyweb on
freenode.
>> I've offered to throw money at the problem if someone's willing to take
>> it on btw... not much, but some money. I've talked to a couple of
>> people about that but it stalled in each iteration.... I don't think
>> it's impossible but I guess it's one of those tasks that for whatever
>> reason seems difficult to get going on because there are some
>> complexities around the way Racket builds "collections" that eg don't
>> seem to show up in Python. Anyway that's not a judgement because
>> despite wanting it fairly badly clearly I haven't made progress on it.
>>
>> I have the notes that Dimos wrote up not long ago in case anyone is
>> interested. Dimos, do you mind if I post them to the list?
>>
> Please do share the notes! I can try to hack
> something up \m/\m/.
I looked at the email that Dimos sent to me (also the email I had was
apparently not the most recent email that they are using, corrected in
the addressing this time), and they had asked me if they should post it
to the mailing list, so I think that's consent to post it since they
expressed consideration in doing so in our exchange... so I'm attaching
it.
* Basics of Racket packaging system
Racket provides three abstractions of how to reuse and move around modules of
code. These are:
** Libraries
A library in Racket is a single file module that can be used in other files.
Libraries that are
serving some higher purpose are organised together in Collections.
Documentation can be found
[[https://docs.racket-lang.org/reference/collects.html][here]].
** Collections
Collections are a number of libraries bundled together. They can be added to
the system through
packages. Racket gets informed for presence of collections through collection
link files.
In the filesystem collections are directories that include library files. The
default path they are
stored is #<path:/usr/share/racket/collects>. The corresponding link file is
located at
#<path:/usr/share/racket/links.rktd> and includes a list as illustrated below:
#+begin_src racket
((root "pkgs/racket-lib")
(root "pkgs/main-distribution")
(root "pkgs/2d"))
#+end_src
More specifics about link files and their structure can be found
[[https://docs.racket-lang.org/reference/collects.html#%28tech._collection._links._file%29][here]].
More links can be added to
arbitary directories through 'raco link'. This file informs racket where a
collection resides in the
#<path:/usr/share/racket> directory.
** Packages
Packages in Racket are the abstractions used to share and move around modules
of code. The include a
number of libraries in a collection, or more collections. They are the means
through which
dependencies are defined. The main entry for managing them is 'raco pkg'.
The tool for installing packages is 'raco pkg install' that takes as an
argument a package source
(name of package in a catalog, directory, tar or zip file etc). How it will act
is defined by the
"info.rkt" file included in the package source.
Documentation for
[[https://docs.racket-lang.org/pkg/cmdline.html#%28part._raco-pkg-install%29]['raco
pkg install']] and
[[https://docs.racket-lang.org/pkg/metadata.html]["info.rkt"]].
Racket can be configured for where to install and search for packages through a
confing file
#<path:/etc/usr/racket/config.rktd>. The documentation for the configurations
can be found
[[https://docs.racket-lang.org/raco/config-file.html?q=raco][here]].
* Racket build system
I will get into the thoughts that I have for the various phases of the build
procedure:
** unpack phase
Here normal procedures form 'gnu-build-system' work just fine. The only
exception is the handling of
.plt files, which can be done as the .gem files are handled in
'ruby-build-system'.
** bootstrap/build/install phase
Since building and installing is done with 'raco pkg install', it makes sense
to have a single phase
dealing with this. The problems that exist here are:
1. <#path:/etc/racket/config.rktd> needs to be updated with all the the places
it needs to look for
the packages. These include the following:
1. lib-search-dirs
2. lib-dir
3. bin-dir
4. links-search-files
5. pkgs-search-dirs
6. collects-search-dirs
7. doc-search-dirs
This needs to occur two times. First time to build the package, including
just its inputs. Then
after/while installing the package, we need to create a new config.rktd that
includes the proper
places for all formerly and newly installed packages.
2. Racket packages have circular dependencies.
3. Racket tries to rebuild the dependencies for the package it installs, even
when not needed, based
on timestamps. There exists an environmental variable
"PLT_COMPILED_FILE_CHECK", that in
documentation says if it's set on exist this won't happen. This doesn't seem
to work as intended.
4. Documentation is created by mutating the docs structure and adding new links
to new nodes of documentation.
For the *first issue*, Claes Wallin that worked on racket2nix, in fact
recreates the racket structure
in the outputs folder by copying in the /usr/share/collects, and /lib/racket
folders, and creating a
symlink of /usr/share/racket/include. He then creates uses the store provided
binaries of
racket(racket, raco, gracket) with -G flag to set the config directory in the
outputs and -X flat to
set the collects directory. This way the packages are built and installed in
the racket
configuration we created at the outputs. Then the config.rktd needs to be
generated. Claes Wallin
does this with a racket script, which we could use, but could also just use a
guile function.
I have implemented this, and works as expected, in most cases that don't meet
the following issues.
The *second issue* is resolved by the importer in racket2nix. The derivations
are expanded to
include circular dependencies and while building the offending packages, dummy
inputs are
created. We can do the same using some DFS guile package. Are there any
standard graph libraries in
guile?
The *third issue* is more complicated. Racket will try to to recompile
dependencies based on
timestamps. This creates issues during the build. In racket2nix there is
created a separate
environment where all dependencies are writable. It is an ugly workaround, but
seems to work.
As for the *fourth issue* I didn't research the racket documentation system a
lot, especially since
it's packages are the of the main offenders in circulars dependencies.
Binaries of racket packages are placed in "launchers" in the racket
configuration folder, so
symlinks to '/bin' shall be created.
** testing phase
It's mostly trivial with 'raco test'.
* Racket importer
Racket packages are defined with "info.rkt" files. Since it's sexp-based it's
trivial to parse them
and create package-definitons in guile. There exists some edge-cases like
'implies' that change the
way the packages are upgraded, but in general it pretty simple.
Here is also the place where maybe we could solve the thing with circular
dependencies. Is it
possible to create custom fields for inputs in guix derivations?
- Racket will move on top of Chez soon, Christopher Lemmer Webber, 2020/10/17
- Re: Racket will move on top of Chez soon, Pierre Neidhardt, 2020/10/17
- Re: Racket will move on top of Chez soon, Bonface M. K., 2020/10/18
- Racket packages / build system, Christopher Lemmer Webber, 2020/10/19
- Re: Racket packages / build system, Bonface M. K., 2020/10/19
- Re: Racket packages / build system,
Christopher Lemmer Webber <=
- Re: Racket packages / build system, Dimos Dimakakos, 2020/10/19
- Re: Racket packages / build system, Bonface M. K., 2020/10/19
- Re: Racket packages / build system, Christopher Lemmer Webber, 2020/10/20
- Re: Racket packages / build system, Ludovic Courtès, 2020/10/21
- Re: Racket packages / build system, Bonface M. K., 2020/10/21