[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Conda environments and reproducibility

From: Thibault Lestang
Subject: Conda environments and reproducibility
Date: Mon, 28 Nov 2022 17:28:48 +0000
User-agent: mu4e 1.6.10; emacs 28.1

Hi there,

I'm new to the list so apologies if this has already been discussed

I've been ruminating on reproducible builds ever since I attended the 10
years birthday event a few months ago. For me, putting /python/ and
/reproduciblity/ together in the same sentence used to invariablity lead
to /virtualenvs/ or /conda/ being featured in the next - suffice it to
say that part of my world view was shattered a bit (that's okay).

Things progressively start to make sense, but when talking about Guix
with a colleague earlier today it became apparent that my understanding
isn't exacly rock solid yet. Particularly, looking at this tweet

referred to in Ludovic's article "Toward reproducible Jupyter notebooks"

The tweet says (22 Jan 2019)

Me, 6 months ago: I am going to save this conda
environment with all the versions of all the packages so it can be
recreated later; this is Reproducible Science!

conda, today: these versions don't work together, lol.

I simply can't explain how such a behavior can happen.

I understand that conda ships pre-compiled binaries. I see how that's
bad for reproducibility and provenance tracking since it's not
straightforward to know how these binaries and dependencies were
compiled. I'm assuming that, when conda saves an environment, it records
version tags and "everything else required" to pull the same binaries
later. Okay - I see how binaries could /technically/ be modified at a
later stage whilst maintaning the same version tag (provenance tracking

Is it the case that someone at Anaconda would modify some package,
keeping the same version tag and other identifiers used by conda, whilst
at the same time marking this package as incompatible with packages it
was previously compatible with?

Thanks for reading!


Dr Thibault Lestang
Senior Research Software Engineer
Department of Aeronautics, Imperial College London

reply via email to

[Prev in Thread] Current Thread [Next in Thread]