Re: Guix Workflow Language ?

[zimoun]

Dear Roel,

Thank you for your comments.

I was imaging your point 2. And the softwares come from Guix.
The added benefit was: a controlled and reproducible environment.
In other words, the added benefit came from the GuixWorkflow (the
engine of workflow), and not from the Language (lisp EDSL).
But maybe it is a wrong way.

>From my experience, the classical strategy of writing pipelines is to
adapt an already existing workflow for one another particular
question. We fetch bits here and there, do some ugly and dirty hacks
to have some results; then depending on them, a cleaner pipeline is
written (or not! :-) or other pieces are tested.
Again from my experience, there is (at least) 3 issues: the number of
tools to learn and know enough to be able to adapt; the bits/pieces
already available; the environment/dependencies and how they are
managed.

In this context, since 'lispy' syntax is not mainstream (and will
never be), it appears to me as a hard position. That's why I asked if
a Guix-backend workflow engine for CWL specs is doable. Run CWL specs
workflow on the top of the GWL engine.

However, I got your point, I guess.
You mean: it is a lot of work with unclear benefits over existing engines.


Therefore, your point 1. reverses "my issue".
Once the pipeline is well-established, write it with GWL! :-)
Next, if it is possible to convert this GWL specs pipeline to CWL one
[+ Docker] (with softwares coming from Guix), then we can enjoy the
CWL-world engine capabilities.
The benefit of that is from two sides: run the pipeline with different
engines; and produce a clean docker image.

So , instead of working on improving the GWL engine (adding features
about efficiency, Grid,  Amazon, etc.) which is a very tough task, the
doable plan would be to add an "exporter".
Right ?


Another question, do you think it is doable to write "importers" ?

I am not sure that the metaphor is good enough, but do you think it is
a feasible goal from the existing GWL to go towards a kind of `Pandoc
of workflows` ? also packing the softwares.

And a start should be:
 - write a parser for (subset of) CWL yaml file and obtain the GWL
representation of the workflow
 - write a exporter to CWL + Docker image

What do you think ?


About the parser, I haven't found yet an easy-to-use Guile lib for
parsing YAML-like files. Any pointer ? Adapt some Racket ones ?



Thank you for your insights.

All the best,
simon