This happens on Linux machines (maybe others) when the machine runs out of
memory. If that's the case, you should see an entry in your syslog talking
about oom-killer. Check your syslog in any case.
I was encountering this recently on an Ubuntu 16.04 machine. There was a
kernel bug where the kernel would think it was out of memory, and start
killing processes, even though there was plenty of swap. See
https://bugs.launchpad.net/bugs/1655842. It is fixed in recent updates
(you will see people there claiming it is not fixed, but I think they are
either finding different bugs or they really are out of memory). At least,
since updating it no longer happens for me.
On Tue, 18 Apr 2017, Russell Clemings via Duplicity-talk wrote:
This looks like the same problem discussed last summer in
https://lists.nongnu.org/archive/html/duplicity-talk/2016-
07/msg00005.html
but I can't see that there was any resolution.
Full (but not incremental) backups are (apparently at least) failing with
no traceback, just the word "Killed," and the lockfile is not removed.
I've pasted the last lines below from the most recent attempt.
It seems to be failing at same place in the backup every time (after
var/spool/mail/wcsj2017). But the "Processing local manifest" filename is
different each time.
It worked fine until a few days ago. And as I said, incremental backups
are
fine (although the full backup failure leaves a lock file that has to be
removed; same filename each time). Running collection-status yields the
results I expect -- a list of full backups and associated incrementals,
and
"No orphaned or incomplete backup sets found." So maybe the backup is
succeeding despite the "Failed" message and lock problem.
I'm not seeing anything that would suggest a memory or load issue. Just in
case, I've run it at different times of day as well as in the middle of
the
night, with the same results.
Suggestions? Should I maybe clean out
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d on the theory
that
the backend is fine but the local cache is corrupted?
A var/spool/mail/swnet
A var/spool/mail/wcsj17dv
A var/spool/mail/wcsj2017
AsyncScheduler: running task synchronously (asynchronicity disabled)
Writing duplicity-full.20170418T140701Z.vol115.difftar.gpg
Uploading s3://
s3.amazonaws.com/BUCKETNAME/duplicity-full.20170418T140701Z.
vol115.difftar.gpg
to STANDARD Storage
Deleting
/home/newton/duplicity-0.7.12/s3backuptmp/duplicity-7y1aYN-t
empdir/mktemp-d8rFt2-116
AsyncScheduler: task completed successfully
Processed volume 115
Writing duplicity-full-signatures.20170418T140701Z.sigtar.gpg
Uploading s3://
s3.amazonaws.com/BUCKETNAME/duplicity-full-signatures.201704
18T140701Z.sigtar.gpg
to STANDARD Storage
Deleting
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full-signatures.20170418T140701Z.sigtar.gpg
Writing duplicity-full.20170418T140701Z.manifest.gpg
Uploading s3://
s3.amazonaws.com/BUCKETNAME/duplicity-full.20170418T140701Z.manifest.gpg
to
STANDARD Storage
Deleting
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170418T140701Z.manifest.gpg
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170128T070703Z.manifest
(28635198)
Found 97 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170204T070703Z.manifest
(28771709)
Found 99 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170211T070707Z.manifest
(28798317)
Found 100 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170218T070703Z.manifest
(28785755)
Found 102 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170225T070705Z.manifest
(29127918)
Found 104 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170304T070703Z.manifest
(29236665)
Found 106 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170311T070712Z.manifest
(29239486)
Found 107 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170318T060703Z.manifest
(29246628)
Found 109 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170325T060703Z.manifest
(29266901)
Found 111 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170401T060704Z.manifest
(29375862)
Found 112 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170401T151402Z.manifest
(29378174)
Found 112 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170408T060702Z.manifest
(29085454)
Found 113 volumes in manifest
Processing local manifest
/root/.cache/duplicity/5c439fc8a10be3c159ce21e965eef31d/
duplicity-full.20170415T060703Z.manifest
(28559242)
/home/newton/duplicity-0.7.10/s3backup/backup_full.sh: line 97: 9126
Killed /home/newton/duplicity-0.7.12/bin/duplicity full
--verbosity 8 --timeout=30 --num-retries=5 --volsize=250
--tempdir=/home/newton/duplicity-0.7.12/s3backuptmp
--encrypt-key=${GPG_KEY} --sign-key=${GPG_KEY}
--exclude=/home/newton/duplicity-0.7.10/s3backup
--exclude=/home/newton/duplicity-0.7.12/s3backup --exclude=/home/.cpan
--exclude=/home/.cpcpan --exclude=/home/virtfs --exclude=/root/.cache
--exclude=/proc --exclude=/dev --exclude=/home/scie2010/public_html/media
--exclude=/home/mailarch-nasw/archives --exclude=/etc/csf/csf.deny
--exclude=/etc/udev/devices/ploop54946p1
--exclude=/etc/valiases/nasw_aliases
--exclude=/home/cybraria/.spamassassinenable
--exclude=/home/sciencew/public_html/stats/logs
--exclude=/root/.cpanel/caches/dynamicui --exclude=/root/.MirrorSearch/
myip.cpanel.net --exclude=/root/tmp --exclude=/usr/local/apache/domlogs
--exclude=/usr/local/apache/logs --exclude=/usr/local/cpanel/3r
dparty/share
--exclude=/usr/local/cpanel/base/frontend
--exclude=/usr/local/cpanel/img-sys --exclude=/usr/local/cpanel/logs
--exclude=/usr/local/cpanel/src --exclude=/usr/local/cpanel/src/nsswitch
--exclude=/usr/local/cpanel/src/redhat_pathtools
--exclude=/usr/local/cpanel/src/remove_sendmail_rpm --include=/etc
--include=/home/clamav --include=/home/cybraria --include=/home/devnasw
--include=/home/mailarch-nasw --include=/home/newton
--include=/home/scie2008 --include=/home/scie2009 --include=/home/scie2010
--include=/home/scie2011 --include=/home/scie2012 --include=/home/scie2013
--include=/home/scie2014 --include=/home/scie2015 --include=/home/scie2016
--include=/home/scie2017 --include=/home/scie2018 --include=/home/scie2019
--include=/home/scie2020 --include=/home/scie2021 --include=/home/scie2022
--include=/home/sciencew --include=/home/solr --include=/home/sw2016dv
--include=/home/swmeeting --include=/home/swnet --include=/home/wcsj17dv
--include=/home/wcsj2017 --include=/root --include=/s3backup
--include=/scripts --include=/usr/local --include=/var/lib/mysql
--include=/var/mail --include=/var/named --include=/var/spool
--exclude=/**
${SOURCE} ${DEST}
--
Nate Eldredge
address@hidden