Re: [gwl-devel] support for containers

[Ricardo Wurmus]

Ricardo Wurmus <address@hidden> writes:

> the GWL could already support execution in containers with this patch:
[…]
> The directories to be mounted in the container are derived from the
> declared inputs and outputs.  The only problem is that inputs are
> read-only in this implementation.  I like it this way, actually, but it
> means that the extended example workflow won’t work as it tries to
> delete its inputs.
>
> Should data inputs be declared as (mutable-file …) or (file …) instead
> of being plain strings?

Taking a step back I think it is worth remembering that ultimately we’d
like to have output caching via an immutable data store.  I think
declaring files as mutable would be a mistake.  Garbage collection is
better than modifying output files.

Some thoughts on how the data store should work: it’s easy to add stuff
to the store (we just hash the inputs leading up to the output,
excluding the output itself); but how do we make store items available
to a process?  When using containers we can bind mount the file to the
declared input location; when not using containers we need to link or
copy the file from

I also want to have finer control over inputs.  Only declared input
*files* should be available in the container, not whole *directories*.

--
Ricardo