Re: [Savannah-help-public] backup tape space

[Ward Vandewege]

Hi Sylvain,

On Wed, May 17, 2006 at 08:22:41AM +0200, Sylvain Beucler wrote:
> Can you tell us some more about it? Are there incremental backups?

Currently we do a daily rsync, and once a week a full backup to tape. We
don't do incremental backups.

> What makes it difficult to restore a single file? 

The sheer amount of the data. If you want to restore a single file that
happens to be on tape 4 or 5 (each tape holds 100GB without compression), we
need to spool through all the tapes until it is found. This takes the better
part of a day.

> What's the status of the plans to improve it? 

We have plans but nothing concrete yet. We are currently very busy with the
new mail system, as well as a lot of other things. Right now, I'm trying to
remove things from the backups that don't need to be backed up - for instance
the /root/Trash folder on Savannah.

> Is there documentation on this for us to read?

Not really.

> > Currently, the total backup set is about 666GB. The Savannah backups are
> > almost 200GB of that. Obviously, the smaller the backup set, the faster
> > backups are finished, and the easier it becomes to restore things from tape.
> 
> Is the backup set a series of physical tapes? 

Yes.

> Does that count incremental backup? 

We don't do incremental backups right now.

> (since Savannah currently uses 188GB I'm a bit puzzled by "200GB").

I said 'almost 200GB'; I was being lazy. If you do 188 * 1024 * 1024 you will
see that that number is almost 200.000.000, which is what the du -s command
told me.

> > Are there old files on Savannah that are not necessary anymore? If so, could
> > you remove them? Or if there are large directories that should not be backed
> > up, could you tell me their paths so that I can exclude them from the rsync?
> > 
> > For instance, the /root/Trash folder is 100GB (!). I assume the contents of
> > this folder is well, trash. Can something be done about this?
> 
> You can probably exclude /root/Trash from the backup.

I have now. That should bring us down to 4 tapes again - we were up to 5 and
even 6 earlier this year.

Can you also clean out /var/tmp? It has a lot of large files from 2004.

> Can you also clean-up the /root/kernel directory? I have no idea
> whether it can be removed or not. I moved all kernel-related files
> there, but they originate from FSF admins :)

Very good, I will make sure that that happens.

> Beware of remounted filesystems. /home is ~140MB and is mounted 4
> times. 

OK; I have excluded 3 of those mounts.

> You can also exclude the cvs ramfs temporary directories. 

I've added those too.

> The
> mount points may change though, so we need to decide what to do in
> such case (do you build the exclude list by parsing /etc/fstab, do we
> notify you when those mount point changes, should we share a doc on
> this somewhere...?)

It's just a file right now. It's probably best if you just e-mail
address@hidden when the mountpoints change, and ask us to update the backup
configuration.

For your information, this is the current 'exclude' file for Savannah. If you
have suggestions for things that should be added/removed from this list, let
us know.

/savannah/dev
/savannah/var/run/zope/default/pcgi.sock
/subsystems/arch/var/run/apache2/cgisock
/subsystems/savannah/var/run/mysqld/mysqld.sock
/var/run/mysqld/mysqld.sock
/savannah/tmpfs/
/root/Trash/
/initrd/
/subsystems/accounts/proc
/subsystems/accounts/dev
/subsystems/accounts/tmp
/subsystems/apt-proxy/proc
/subsystems/apt-proxy/dev
/subsystems/apt-proxy/tmp
/subsystems/arch/proc
/subsystems/arch/dev
/subsystems/arch/tmp
/subsystems/cvs/proc
/subsystems/cvs/dev
/subsystems/cvs/tmp
/subsystems/cvs/var/lock/cvs
/subsystems/compilation-environment/proc
/subsystems/compilation-environment/dev
/subsystems/compilation-environment/tmp
/subsystems/download/proc
/subsystems/download/dev
/subsystems/download/tmp
/subsystems/savannah/proc
/subsystems/savannah/dev
/subsystems/savannah/tmp
/savannah/proc
/subsystems/cvs/cvsroot.sv/*/dev/
/subsystems/cvs/var/run/apache2/cgisock
/subsystems/download/var/run/apache2/cgisock
/var/bind/dev
/var/tmp
/root/deleted-projects/*/cvsroot/dev/null
/subsystems/arch/home
/subsystems/cvs/home
/subsystems/download/home

Thanks,
Ward.

-- 
Ward Vandewege <address@hidden>
Free Software Foundation - Senior System Administrator