[bug#39547] [PATCH] website: Provide JSON sources list used by Software Heritage.

[zimoun]

Hi Ludo,

On Tue, 18 Feb 2020 at 09:24, Ludovic Courtès <address@hidden> wrote:
> zimoun <address@hidden> skribis:
> > On Fri, 14 Feb 2020 at 11:20, Ludovic Courtès <address@hidden> wrote:
> >> zimoun <address@hidden> skribis:

> >> >> Could you, in a first patch, move ‘origin->json’ out of
> >> >> ‘packages-json-builder’, and in a second patch, add mirror-expansion
> >> >> feature?
> >> >
> >> > Yes, I will try but I am a bit lost. There is:
> >> >  - packages-json-builder that I did not modified
> >> >  - sources-json-builder which the "adaptation" of the former to output
> >> > to the sources.json format.
> >> >
> >> > Well, do you want a refactor of 'origin->json' shared by the 2
> >> > "{sources,packages}-json-builder"?
> >>
> >> Yes, exactly.  No reason to have two copies of that code.
> >
> > The minor "issue" is the 'match' piece of 'origin->json':
> >
> >                `(("url" . ,(match uri
> >                              ((? string? url) (vector url))
> >                              ((urls ...) (list->vector urls))))))
> >
> >
> > Because in the 'sources.json' it is always one unique string.
> > However, in 'packages.json' it is always a vector of one or several
> > elements. And in the previous patch, the job of 'resolve' was to
> > uniquify these elements for 'sources.json'.
>
> Ah, good point.  What about adding a keyword parameter to ‘origin->json’
> that would tell whether to return a single URL or a list thereof?

Sharing the same code.

 - packages.json: as it is now.
 - sources.json: expand mirror and resolve to display only URL.

I am going for that.


> > Just to be on the same wavelength, does 'packages.json' stay as it is
> > (vector for the source url field)?
>
> I think ‘packages.json’ should still return a list of URLs.

Ok.


> I’ve also added a comment asking whether ‘sources.json’ could be changed
> to contain a list of URLs:
>
>   https://forge.softwareheritage.org/D2025#64037

I have seen. :-)
Even, all the mirrors list should be expanded, not only the first one.


> > Or is 'packages.json' modified to record only one url for the source
> > url field with mirror:// expanded?
>
> Yeah, I think we can expand mirror URLs.

So, 'mirror://' will become a list (from '%mirrors') of URLs, right?


Cheers,
simon