[bug#33265] [WIP RFC v4] services: Add file system monitoring service.

[Danny Milosavljevic]

Hi Ludo!

On Sat, 10 Nov 2018 23:19:53 +0100
address@hidden (Ludovic Courtès) wrote:

> Nice!  Should we call it ‘fswatch-monitoring-service’, since there may
> be other tools to achieve similar results?

Its use is mostly to allow other Guix services to extend it in order to
do stuff on file change (for example restarting a service; starting the
user shepherd - the latter is what caused me to write this).

I was very much tempted to call it "fswatch-monitoring-service", but that's
really an implementation detail - what it does is provide file change
notification services to other Guix services.

It's not meant to be a generic fswatch service.  I've not known fswatch
before writing this service - so it's pretty likely that there are tons
more features an fswatch user could want to use which the service doesn't
provide.  Also, fswatch isn't really meant to be used as a daemon AFAIK,
so we are not providing fswatch to the user in the way it's supposed to be
used by him.

> The ‘sleep’ call looks suspicious.  :-)

Pretty suspicious :)

Right now, the service has only a very narrow usecase where rapid
notifications is not something one wants.  On the contrary, both rapid
notifications and (in a pathological case where a watch couldn't be
registered - see the "/does_not_exist" workaround) rapid fswatch invocations
would waste (potentially enormous amounts of) cpu time and would
forkbomb fswatch, shepherd and/or worse things.  On the other hand, using
fswatch like we do now causes us to lose events.  But so does starting fswatch
later than the clients.  And most fswatch backends (for example the Linux one)
can lose events anyway (since the kernel buffer is limited - Linux will then
drop events).  That's why it always calls the handlers when restarting - just
in case.

> IIUC, the service stops (and is respawned) every time an even[t] occurs, is
> that right?

Yes.  It's very simple :)

> Can’t we instead remove ‘--one-event’ and pass fswatch a script to run?

Not to my knowledge.  It would be nice...

What the stuff here does is it waits for events to accumulate in the time
given by "-l" (that is called a "batch" in fswatch).

The first such batch which contains at least one event will cause fswatch
to exit.
Otherwise fswatch will continue waiting (blocking) until the above happens.

It's an explicitly supported way to use fswatch (because it's very resilient).

The service collects all the files from the system into one fswatch
invocation - it's very much supposed to be a tripwire and not something
regularily used.  It also collects all the handlers (regardless of monitored
file) into one handler to be called.

> In that case, we’d pass it something along these lines:
> 
>   #$(scheme-file "event-handler.scm"
>                  #~(for-each (lambda (handler) …) …))
> 
> WDYT?

I'd like to, but it doesn't support it.

What would be possible is to pipe something at the end (i.e. use open-pipe), but
I've had bad experiences with pipes buffering too much in the past and I'd 
rather
not hang the entire thing by some stupid buffering or parsing problem.

That said, we could try in a future version (after some serious testing).
But I think that trying to use pipes for us to be remote-controlled by
fswatch it will not work correctly if fswatch doesn't exit.

It would work with a pty, but that's kinda... convoluted.  Also, forkpty is not
available in Guile.

There's libfswatch but no Guile bindings yet.  Would be cool to have those
because fswatch is cross platform (and it has C bindings).

> > +(define file-system-monitoring-service-type
> > +  (service-type (name 'monitor-file-system)
> > +                (extensions
> > +                 (list (service-extension shepherd-root-service-type
> > +                                          
> > file-system-monitoring-shepherd-services)))
> > +                (compose concatenate)
> > +                (extend (lambda (config monitored-entries)
> > +                          (let ((monitored-files
> > +                                 (map 
> > file-system-monitoring-entry-file-name
> > +                                      monitored-entries))
> > +                                (handlers
> > +                                 (map file-system-monitoring-entry-handler
> > +                                      monitored-entries)))  
> 
> So here if we changes ‘handlers’ to ‘handler’, we could do:
> 
>   (apply compose (map file-system-monitoring-entry-handler entries))

All good points!

pgpbpwmkhT7a9.pgp
Description: OpenPGP digital signature