Re: [Duplicity-talk] duplicity verification and corrupt backups

[edgar . soldin]

On 22.08.2011 04:25, Ed Blackman wrote:
> On Wed, Aug 17, 2011 at 06:15:26PM +0200, address@hidden wrote:
>> Good to know. But seriously. A slim line also minimizes the throughput and 
>> therefor the data you can put through it over a timeframe. Doing a full over 
>> a timeframe of more than a day is challenging at best. I would not advise it.
>>
>> Rather
>>
>> A) split the backup into small parts that are not backed that often
>> or
>> B) do what lot's of people with slow upload channels do. Do duplicity backup 
>> to a local file:// target and rsync or upload it with the software of your 
>> preference to the remote site.
> 
> or
> C) take filesystem snapshots (I use LVM on Linux), then backup from the 
> snapshots.
> 
> The advantage of snapshots over option B is that the snapshots are created in 
> a matter of seconds, and so represent a much more consistent view of the 
> system than even a quick backup to a local file:// target.
> 
> The disadvantage is that there's a significant scripting overhead.  Not only 
> setting up and tearing down the snapshots, but also just interacting with 
> duplicity.  "--rename $snapshotroot /" gets you most of the way, and it 
> wouldn't be an option without it, but you also have to change all the 
> --includes and --excludes (including filelists) to be relative to the root of 
> the snapshot.
> 
> But in the end, it works.  Some of my full backups take 10 days to "trickle" 
> up to Amazon S3, by my script creates the snapshot for it and all the 
> incrementals blocked while the full backup completes.
> 

Still the probability of line reset or something else interrupting duplicity 
uploading will significantly raise the probability of resuming gone wrong or 
corrupt files in general on the backend. I definitely would not advise to have 
duplicity running that long.

Ed: Do you verify your backups from time to time?

ede/duply.net