[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [gwl-devel] support for containers

From: Ricardo Wurmus
Subject: Re: [gwl-devel] support for containers
Date: Tue, 29 Jan 2019 10:38:53 +0100
User-agent: mu4e 1.0; emacs 26.1

Ricardo Wurmus <address@hidden> writes:

> the GWL could already support execution in containers with this patch:
> The directories to be mounted in the container are derived from the
> declared inputs and outputs.  The only problem is that inputs are
> read-only in this implementation.  I like it this way, actually, but it
> means that the extended example workflow won’t work as it tries to
> delete its inputs.
> Should data inputs be declared as (mutable-file …) or (file …) instead
> of being plain strings?

Taking a step back I think it is worth remembering that ultimately we’d
like to have output caching via an immutable data store.  I think
declaring files as mutable would be a mistake.  Garbage collection is
better than modifying output files.

Some thoughts on how the data store should work: it’s easy to add stuff
to the store (we just hash the inputs leading up to the output,
excluding the output itself); but how do we make store items available
to a process?  When using containers we can bind mount the file to the
declared input location; when not using containers we need to link or
copy the file from

I also want to have finer control over inputs.  Only declared input
*files* should be available in the container, not whole *directories*.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]