[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [gwl-devel] support for containers

From: zimoun
Subject: Re: [gwl-devel] support for containers
Date: Wed, 30 Jan 2019 11:17:02 +0100

Hi Ricardo,

On Wed, 30 Jan 2019 at 00:16, Ricardo Wurmus <address@hidden> wrote:

> Since we don’t hash the data (because it’s expensive) the scripts are
> “proxies” for the data files.  We compute the hashes over the dependent
> scripts and assume that this is enough to decide whether to recompute
> data files or to serve them from the cache/store.

Just to be sure to well understand your point, let pick the simple
example from genomics pipeline:
 FASTQ -align-> BAM -variant-> VCF
So, you intend to hash:
 - the data FASTQ
 - the scripts align and variant
Or only the scripts containing reference to inputs (here FASTQ), where
the reference is a location fixed by the user.

Well, hashing the scripts and assuming they "mirror" the data files
appear to me an efficient design for the CAS.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]