Re: Hack the (init) system!

[Thompson, David]

On Thu, Sep 3, 2015 at 5:02 PM, Ludovic Courtès <address@hidden> wrote:
> Howdy Guix!
>
> We’ve all been talking about it for some time: a REPL server in dmd!
>
> This can be done by changing zero lines in dmd:
>
>
>
> Then you can do:
>
>   deco start dmd-repl
>   socat unix-connect:/var/run/dmd/repl stdio
>
> and tinker from there.  New ways to cra^W experiment with your system!
>
> I’m tempted to just commit that.  There are shortcomings: (1) the REPL
> server runs in a thread and threads + fork don’t go together well
> (although in practice dmd only does fork followed by exec, so it’s OK),
> and (2) for some reason ‘stop-server-and-clients!’ seems to leave open
> sockets behind it, so if you restart the REPL on the same socket, it
> fails with EADDRINUSE.
>
> Thoughts?

Awesome little hack.  Thanks!  I probably won't use this for PID 1,
but I will use something similar for my userspace dmd process.

Now, I think this solves the immediate need of being able to live hack
dmd, but I'd like to note an additional shortcoming: Thread
synchronization isssues.  As you know, there's potential for the main
dmd thread and a REPL thread to write to the same memory and blow
things up.  In practice, this would be unlikely, but it shouldn't even
be a possibility.  To accommodate programs that run in an event loop
(though I know dmd doesn't truly have this yet), Mark and I developed
the (system repl coop-server) module available in Guile 2.0.11.  This
"cooperative" REPL server can be integrated into an event loop and
guarantee that expressions are only evaluated in the context of a
single thread, in this case the main thread.  I think that this should
be the approach taken in the not-so-long term, which will require
modifying dmd itself.  I have been using the cooperative REPL server
for my game engine since Guile 2.0.11 was released and I've been able
to comfortably live hack games without fear.  Problem (1) still exists
because threads are  used for each client, but the real magic happens
via coroutines.  Not sure if we can reasonably get rid of threads
altogether and uses a non-blocking I/O model.  Might be worth looking
into, but I'm getting off-topic.

Now that we can live hack dmd, we'll need some things to help make it
pleasant.  Most of the time I am tweaking service definitions until
they are just right.  Currently, that means calling a procedure to
unload the version that exists, and then registering a new one.  I'd
like to reduce that to a single step to tighten the feedback loop.
What do you think about adding a 'register-services*' procedure, or
maybe a 'define-service' form, that first unregisters the old service
before registering the new one?

If we can achieve a safe live hacking environment, then we'll have one
hell of an init system, I say.  Thanks!

- Dave