Re: Accuracy of importers?

[zimoun]

Hi Ludo,

On Fri, 29 Oct 2021 at 23:57, Ludovic Courtès <ludo@gnu.org> wrote:

> (It’s quite expensive to run because it downloads a whole bunch of
> things and tries many 404 URLs in the case of CRAN before finding the
> right one.)

Ah… it requires investigation thus.


> --8<---------------cut here---------------start------------->8---
> $ SAMPLE_SIZE=200 ./pre-inst-env guile 
> ~/src/guix-debugging/importer-accuracy.scm
> […]
> Accuracy for 'pypi' (200 packages):
>   accurate: 58 (29%)
>   different inputs: 142 (71%)
>   different source: 0 (0%)
>   inconclusive: 0 (0%)
> Accuracy for 'cran' (200 packages):
>   accurate: 176 (88%)
>   different inputs: 23 (12%)
>   different source: 1 (0%)
>   inconclusive: 0 (0%)
> --8<---------------cut here---------------end--------------->8---

[...]

> The script doesn’t do anything useful for crates because they have their
> own way of representing inputs.  It doesn’t account for changes in
> ‘arguments’ like zimoun suggested, meaning it’s overestimating
> accuracy.

It is already quite interesting results.  Because it shows upstream
stability, IIUC.  Well, it means that running “guix import pypi” one
months ago and running the sames now, 71% packages have different
inputs.  Right?  It is because some metadata from PyPI changed, right?
Not because “guix import pypi” was doing wrong and now it does better,
right?

IMHO, it shows how PyPI allows bad practises about packaging, isn’t it?

My understanding of this experiment is about upstream “quality”, not
about importer “accuracy”.  Do I incorrectly understand?


Cheers,
simon