Date: Thu, 5 Oct 2017 20:19:23 +0300
From: Mikko Ohtamaa <address@hidden>
To: Discussion about duplicity backup <address@hidden>
Cc: Philip Jocks <address@hidden>, Tom Ekberg <address@hidden>
Subject: Re: [Duplicity-talk] Scalability of duplicity
Hi,
Sorry about the scant environment details. I'm running duplicity 0.7.06
on Ubuntu 16.04.3 with 63GB of RAM and 12TB of disk space. The lscpu command
says
it has 48 CPUs. Big honkin' server. The command 'gpg --version' displays
gpg (GnuPG) 1.4.20. The destination host where duplicity puts the backup files
is
running Ubuntu 16.04.3 with 8GB of RAM and 4.9TB of disk space. The lscpu
command says it has 4 CPUs. 31 days of backups should need about 2.7TB of
storage
without compression and encryption.
If there is no CPU activity and Duplicity is still running it might be that the
underlying network protocol has stalled. Are you moving files over SFTP or
SCP? If the
underlying network hangs and does not properly timeout then it might cause
symptoms like this.
You can use strace system command the check what a process is waiting if it
does nothing.
Hope this gives clues,
-Mikko
Read the duplicity change log down to the 2015-12-07 line that says 'Prep
for 0.7.06'. Didn't see anything specifically there (I didn't look at the bug
DB)
describing a fix for this problem. I could try running the latest
duplicity, 0.7.14, to see if the problem goes away.
What version of duplicity are you running?
Tom Ekberg
Senior Computer Specialist, Lab Medicine
University of Washington Medical Center
1959 NE Pacific St, MS 357110
Seattle WA 98195
work: (206) 598-8544
email: address@hidden
On Thu, 5 Oct 2017, Philip Jocks wrote:
Date: Thu, 5 Oct 2017 11:06:31 +0200
From: Philip Jocks <address@hidden>
To: Discussion about duplicity backup <address@hidden>
Cc: Tom Ekberg <address@hidden>
Subject: Re: [Duplicity-talk] Scalability of duplicity
Hej,
Now for my questions:
Has anyone run duply/duplicity with 90GB of binary files?
yes, we do run it with backup sets >300GB, haven't had any issues
with stalling backups in the last few years as far as I remember. Running on
FreeBSD, though.
Which version of duplicity are you running?
Cheers,
Philip
_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
--
Mikko Ohtamaa
http://opensourcehacker.com
http://twitter.com/moo9000