Re: Using a CDN or some other mirror?

[Thompson, David]

On Tue, Dec 4, 2018 at 9:06 AM Ludovic Courtès <address@hidden> wrote:
>
> Hi Hartmut,
>
> Hartmut Goebel <address@hidden> skribis:
>
> > Am 03.12.2018 um 17:12 schrieb Ludovic Courtès:
> >> Thus, I’m thinking about using a similar setup, but hosting the mirror
> >> on some Big Corp CDN or similar.
> >
> > Isn't this a contradiction: Building a free infrastructure relaying on
> > servers from some Big Corporation? Let allow the privacy concerns
> > raising when delivering data via some Big Corporation.
> >
> > If delivering "packages" works via static data without requiring any
> > additional service, we could ask universities to host Guix, too. IMHO
> > this is a much preferred solution since this is a decentralized publish
> > infrastructure already in place for many GNU/Linux distributions.
>
> As you know, berlin.guixsd.org is hosted at the Max Delbrück Center in
> Berlin, a public research institute.  So in a way, we’re already doing
> that.  We shouldn’t take it for granted that public institutes will
> happily host our stuff and donate hardware: without Ricardo’s work and
> the generosity of the MDC, we wouldn’t have anything there.
>
> I understand the reluctance regarding “Big Corp” hosting, and I actually
> share it to some extent.  However, having put much thought into it (and
> also much sweat in build farm sysadmin…), I think the alternative is:
> commercial hosting, or peer-to-peer.
>
> Florian has been looking at the latter approach with IPFS, and perhaps
> we’ll be able to put it in production in a few months and be happy with
> it (I have good hopes given what Florian already demonstrated.)
>
> In the meantime, we need redundant storage, high bandwidth, and high
> availability.  If you know of non-profit organizations that can provide
> such services, please let us know; if not, we’ll resort to a commercial
> service.  The bottom line is: we cannot reasonably pretend to offer such
> a service ourselves.
>
> (Note that we’re just talking about substitute delivery—I wouldn’t want
> to *build* packages on one of these commercial hosting services.)
>
> I hope this clarifies my position.

Using CloudFront with a custom (non-S3) origin sounds like a
reasonable solution to me, though I understand the hesitance to use a
commercial service.

If AWS CloudFront is the path chosen, it may be worthwhile to follow
the "infrastructure as code" practice and use CloudFormation to
provision the CloudFront distribution and any other supporting
resources. The benefit is that there would be a record of exactly
*how* the project is using these commercial services and the setup
could be easily reproduced.  The timing is interesting here because I
just attended the annual AWS conference on behalf of my employer and
while I was there I felt inspired to write a Guile API for building
CloudFormation "stacks".  You can see a small sample of what it does
here: https://gist.github.com/davexunit/db4b9d3e67902216fbdbc66cd9c6413e

- Dave