Re: OF packages: check OF repositories

[Andrew Janke]

On 4/15/19 11:14 PM, Kai Torben Ohlhus wrote:
> On Tue, Apr 16, 2019 at 7:25 AM Doug Stewart <address@hidden
> <mailto:address@hidden>> wrote:
> 
>     On Mon, Apr 15, 2019 at 4:54 PM Olaf Till <address@hidden
>     <mailto:address@hidden>> wrote:
> 
>         On Sun, Apr 14, 2019 at 03:29:44PM -0400, Nir Krakauer wrote:
>         > Can we make a plan to regularly back up the whole OF tree on
>         another site
>         > to protect against this kind of eventuality?
> 
>         As for being able to restore, all repositories fit well onto a usual
>         local disk. And there is (or was) already a script to automatize the
>         backup of all repositories (a Makefile made by Carnë) in an extra
>         repository named 'octave-forge'. It only needs someone who regularly
>         thinks of it or scripts it as a cron job.
> 
>         Olaf
> 
>     OK I just ran Carne's make file, and have a copy of what is there now.
>     I will try and keep it up to date.
>     Someone else should also do it. then we might be safe. :-)
> 
>     -- 
>     DAS
> 
> 
> Just for the files, Carnës Makefile is available
> from https://sourceforge.net/p/octave/octave-forge/ci/default/tree/ or via
> 
>    hg clone http://hg.code.sf.net/p/octave/octave-forge octave-octave-forge
> 
> After cloning and running `make update` and about 220 MB with backups of
> 121 hg and git repositories from SF are downloaded in that very same
> directory.
> 
> As this backup is not that big, maybe we can run it daily via cron on
> JWEs digitalocean server, where other backups are located as well:
> 
>  
>  
> https://wiki.octave.org/Project_Infrastructure#Hosted_on_jwe.27s_digitalocean.com_account
> 
> Ideally, there was public web access to those backups like with the
> other https://hg.octave.org repositories.
> This backup can be also done on other server I have access to (JWEs
> dreamhost server):
> 
>  
>  
> https://wiki.octave.org/Project_Infrastructure#Hosted_on_jwe.27s_dreamhost.com_account
> 
> But I am afraid that the backup repositories are not public visible there.
> 
> Best,
> Kai

I've written a script to go with Carnë's Makefile that will mirror all
the OF repos to Bitbucket:

https://bitbucket.org/apjanke/octave-octave-forge/branch/bitbucket-sync

It's best used with a separate Bitbucket account so you don't clutter up
your main Bitbucket account with these mirrors, or get them confused
with your working forks of them. You'll need to set up an SSH key and
password for the Bitbucket account. Then you can `make update; make
bbpush` to update the mirror.

Here's what it looks like when they're all mirrored:
https://bitbucket.org/apjanke-octave/

Bitbucket is free (as in beer), the repos are publicly available, and
they have their own backups. This might be a good option.

The programmatic creation of Git repos via Bitbucket's REST API doesn't
seem to be working, so those need to be created manually, but there's
only a dozen of them, so that's not that much work. I've reported this
issue to Bitbucket.

Cheers,
Andrew