From MAILER-DAEMON Sun Jan 23 12:56:26 2005 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Csly4-0006Tg-GZ for mharc-duplicity-talk@gnu.org; Sun, 23 Jan 2005 12:56:08 -0500 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Cslxt-0006QW-9P for duplicity-talk@nongnu.org; Sun, 23 Jan 2005 12:55:59 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Cslxe-0006Kw-CO for duplicity-talk@nongnu.org; Sun, 23 Jan 2005 12:55:44 -0500 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Cslxe-0006GG-5e for duplicity-talk@nongnu.org; Sun, 23 Jan 2005 12:55:42 -0500 Received: from [80.91.229.2] (helo=main.gmane.org) by monty-python.gnu.org with esmtp (Exim 4.34) id 1Cslao-0006AM-MD for duplicity-talk@nongnu.org; Sun, 23 Jan 2005 12:32:06 -0500 Received: from root by main.gmane.org with local (Exim 3.35 #1 (Debian)) id 1Cslan-0003QK-00 for ; Sun, 23 Jan 2005 18:32:05 +0100 Received: from offby1.atm01.sea.blarg.net ([206.124.138.125]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 23 Jan 2005 18:32:05 +0100 Received: from offby1 by offby1.atm01.sea.blarg.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 23 Jan 2005 18:32:05 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: duplicity-talk@nongnu.org From: Eric Hanchrow Date: Sat, 22 Jan 2005 19:13:10 -0800 Lines: 37 Message-ID: <874qh8bzy1.fsf@offby1.atm01.sea.blarg.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: offby1.atm01.sea.blarg.net User-Agent: Gnus/5.1007 (Gnus v5.10.7) Emacs/21.3.50 (gnu/linux) Cancel-Lock: sha1:P/K7EfijySu/4DjO+AKtlUswDfw= Sender: news Subject: [Duplicity-talk] --remove-older-than: what obvious thing am I missing? X-BeenThere: duplicity-talk@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: duplicity-talk.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2005 17:56:02 -0000 I've use this script to do periodic unattended backups to a couple of remote hosts to which I've got ssh access. #!/bin/sh # Is today Sunday? if test `date +%w` = 0; then howmuch=--full else howmuch=--incremental fi for host in one.host.com another.host.net etc.you.get.the.picture do echo $host ... duplicity \ $howmuch \ --remove-older-than $(date +%s --date='last week') \ --include $HOME/one-important-directory \ --include $HOME/another-important-directory \ --exclude '**' \ $HOME \ scp://$host/dups done It seems to work fine (i.e., the "dups" directories on those hosts are filling up nicely with impressive-looking backup files) except for one thing: those backup files never get deleted. I'd thought that by specifying --remove-older-than, files older than a week would get pruned, but it ain't happening. Am I misunderstanding something? This is version 0.4.1-8 on Debian Sarge (aka "Testing"). -- [Dijkstra's] great strength is that he is uncompromising. It would make him physically ill to think of programming in C++. -- Donald E. Knuth From MAILER-DAEMON Sun Jan 23 14:35:07 2005 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1CsnTR-0007Er-BU for mharc-duplicity-talk@gnu.org; Sun, 23 Jan 2005 14:32:37 -0500 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1CsnTG-00079p-O5 for duplicity-talk@nongnu.org; Sun, 23 Jan 2005 14:32:29 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1CsnT5-00071u-Eu for duplicity-talk@nongnu.org; Sun, 23 Jan 2005 14:32:18 -0500 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1CsnT4-00071V-Ne for duplicity-talk@nongnu.org; Sun, 23 Jan 2005 14:32:14 -0500 Received: from [66.18.174.3] (helo=server.3lefties.com) by monty-python.gnu.org with esmtp (Exim 4.34) id 1CsnEO-0005bB-RQ for duplicity-talk@nongnu.org; Sun, 23 Jan 2005 14:17:05 -0500 Received: from [192.168.1.166] (waldo.3lefties.com [66.18.174.195]) by server.3lefties.com (8.9.2/8.9.2) with ESMTP id MAA22641; Sun, 23 Jan 2005 12:14:21 -0700 (MST) Message-ID: <41F3F826.8000600@3lefties.com> Date: Sun, 23 Jan 2005 12:16:54 -0700 From: Dan Parrish User-Agent: Mozilla Thunderbird 1.0 (X11/20041208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Eric Hanchrow Subject: Re: [Duplicity-talk] --remove-older-than: what obvious thing am I missing? References: <874qh8bzy1.fsf@offby1.atm01.sea.blarg.net> In-Reply-To: <874qh8bzy1.fsf@offby1.atm01.sea.blarg.net> X-Enigmail-Version: 0.89.5.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: duplicity-talk@nongnu.org X-BeenThere: duplicity-talk@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: duplicity-talk.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2005 19:32:33 -0000 Eric Hanchrow wrote: > I've use this script to do periodic unattended backups to a couple of > remote hosts to which I've got ssh access. > > #!/bin/sh > > # Is today Sunday? > if test `date +%w` = 0; then > howmuch=--full > else > howmuch=--incremental > fi > > for host in one.host.com another.host.net etc.you.get.the.picture > do > echo $host ... > duplicity \ > $howmuch \ > --remove-older-than $(date +%s --date='last week') \ > --include $HOME/one-important-directory \ > --include $HOME/another-important-directory \ > --exclude '**' \ > $HOME \ > scp://$host/dups > done > > It seems to work fine (i.e., the "dups" directories on those hosts are > filling up nicely with impressive-looking backup files) except for one > thing: those backup files never get deleted. I'd thought that by > specifying --remove-older-than, files older than a week would get > pruned, but it ain't happening. Am I misunderstanding something? > > This is version 0.4.1-8 on Debian Sarge (aka "Testing"). > When I use the "--remove-older-than 10D" switch, this is what the output says: There are backup set(s) at time(s): Sun Dec 12 20:06:02 2004 Mon Dec 13 04:53:01 2004 Tue Dec 14 04:53:01 2004 Wed Dec 15 04:53:01 2004 Thu Dec 16 04:53:01 2004 Fri Dec 17 04:53:01 2004 Sat Dec 18 04:53:01 2004 Sun Dec 19 04:53:01 2004 Mon Dec 20 04:53:01 2004 Tue Dec 21 04:53:01 2004 Wed Dec 22 04:53:01 2004 Thu Dec 23 04:53:01 2004 Sat Dec 25 04:53:02 2004 Sun Dec 26 04:53:01 2004 Mon Dec 27 04:53:01 2004 Tue Dec 28 04:53:02 2004 Wed Dec 29 04:53:01 2004 Thu Dec 30 04:53:01 2004 Fri Dec 31 04:53:01 2004 Sat Jan 1 04:53:01 2005 Sun Jan 2 04:53:01 2005 Mon Jan 3 04:53:01 2005 Tue Jan 4 04:53:01 2005 Wed Jan 5 04:53:01 2005 Thu Jan 6 04:53:01 2005 Fri Jan 7 04:53:01 2005 Sat Jan 8 04:53:01 2005 Mon Jan 10 04:53:01 2005 Tue Jan 11 04:53:01 2005 Wed Jan 12 04:53:01 2005 Which can't be deleted because newer sets depend on them. No old backup sets found, nothing deleted. IOW, it doesn't prune like rdiff-backup does. rdiff-backup keeps a current full copy and reverse diffs going back in time, while duplicity STARTS with a full backup and tracks the changes along the way. Because of the way it tracks the changes, it can't prune the older tarballs. I haven't figured out a way around this...There might be one, but I haven't read about it in my (admittedly light) reading of the man page and such. As I figure it, your backup directory continues to get larger and larger over time, until you make a completely new backup ***OR*** you delete all increments since the last full backup and start building new incrementals. Anyone else got this figured out? I really liked how rdiff-backup handles everything...It's just not encrypted. -Dan From MAILER-DAEMON Thu Jan 27 07:59:31 2005 Received: from mailman by lists.gnu.org with archive (Exim 4.43) id 1Cu9FD-0000Gj-I6 for mharc-duplicity-talk@gnu.org; Thu, 27 Jan 2005 07:59:31 -0500 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Cu9FB-0000GS-UI for duplicity-talk@nongnu.org; Thu, 27 Jan 2005 07:59:30 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Cu9FB-0000GG-E8 for duplicity-talk@nongnu.org; Thu, 27 Jan 2005 07:59:29 -0500 Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Cu9De-00085u-8p for duplicity-talk@nongnu.org; Thu, 27 Jan 2005 07:57:54 -0500 Received: from [195.60.10.180] (helo=tuxy.concrete-cows.org.uk) by monty-python.gnu.org with esmtp (TLSv1:DES-CBC3-SHA:168) (Exim 4.34) id 1Cu8pf-0002L9-Pj for duplicity-talk@nongnu.org; Thu, 27 Jan 2005 07:33:08 -0500 Received: from [195.60.29.178] (helo=[192.168.1.50]) by tuxy.concrete-cows.org.uk with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.44) id 1Cu8pa-0000EP-T9 for duplicity-talk@nongnu.org; Thu, 27 Jan 2005 12:33:03 +0000 Message-ID: <41F8DF78.6060509@spikings.com> Date: Thu, 27 Jan 2005 12:32:56 +0000 From: Peter Spikings User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: duplicity-talk@nongnu.org X-Enigmail-Version: 0.90.0.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig70B07CA2FE2EC3C7E1D423EF" X-SA-Exim-Connect-IP: 195.60.29.178 X-SA-Exim-Mail-From: peter@spikings.com X-SA-Exim-Scanned: No (on tuxy.concrete-cows.org.uk); SAEximRunCond expanded to false Subject: [Duplicity-talk] --remove-older-than problem X-BeenThere: duplicity-talk@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: duplicity-talk.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 12:59:30 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig70B07CA2FE2EC3C7E1D423EF Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, Sorry if this has been asked before (seems to be a different problem than the one mentioned previously) but I'm having a problem with the above option. Basically it only deletes the signature files on the remote server that are too old, leaving the manifests and the large full backups. Some output to illustrate... First I do a full backup (of a test directory): root@tuxy pjs # duplicity -v9 -f --remove-older-than now --encrypt-key A7C00F42 /home/pjs/a scp://myuser@myhost//usr/home/myuser/etc Main action: full Reading results of 'ssh myuser@myhost ls /usr/home/myuser/etc' Collection Status ----------------- Backend: Archive dir: None Matched pair: None All backup chains: [] Other backup chains: [] Other sig chains: [] Orphaned sig names: [] Orphaned backup sets: [] Incomplete backup sets: [] ----------------Backup Chains---------------------- PASSPHRASE variable not set, asking user. GnuPG passphrase: Selecting /home/pjs/a Comparing () and None Getting delta of (() /home/pjs/a dir) and None Generating delta - new file: . Selecting /home/pjs/a/c Comparing ('c',) and None Getting delta of (('c',) /home/pjs/a/c reg) and None Generating delta - new file: c Running 'scp /tmp/tmpfhYfIf myuser@myhost:/usr/home/myuser/etc/duplicity-full.2005-01-27T12:09:31+01:00.vol1.difftar.gpg' tmpfhYfIf 100% 741 0.7KB/s 00:00 Deleting /tmp/tmpfhYfIf Running 'scp /tmp/tmpEhOZX6 myuser@myhost:/usr/home/myuser/etc/duplicity-full.2005-01-27T12:09:31+01:00.manifest.gpg' tmpEhOZX6 100% 735 0.7KB/s 00:00 Deleting /tmp/tmpEhOZX6 Running 'scp /tmp/tmpQm_uc4 myuser@myhost:/usr/home/myuser/etc/duplicity-full-signatures.2005-01-27T12:09:31+01:00.sigtar.gpg' tmpQm_uc4 100% 766 0.8KB/s 00:00 Deleting /tmp/tmpQm_uc4 Reading results of 'ssh myuser@myhost ls /usr/home/myuser/etc' Ignoring file 'duplicity-full-signatures.2005-01-27T12:09:31+01:00.sigtar.gpg' --------------[ Backup Statistics ]-------------- ------------------------------------------------- Then I repeat hoping that it will delete the previous backup: root@tuxy pjs # duplicity -v9 -f --remove-older-than now --encrypt-key A7C00F42 /home/pjs/a scp://myuser@myhost//usr/home/myuser/etc Main action: full Reading results of 'ssh myuser@myhost ls /usr/home/myuser/etc' Ignoring file 'duplicity-full-signatures.2005-01-27T12:09:31+01:00.sigtar.gpg' Collection Status ----------------- Backend: Archive dir: None Matched pair: (, ) All backup chains: [] Other backup chains: [] Other sig chains: [] Orphaned sig names: [] Orphaned backup sets: [] Incomplete backup sets: [] ----------------Backup Chains---------------------- BackupChain: Fullset: [duplicity-full.2005-01-27T12:09:31+01:00.manifest.gpg, duplicity-full.2005-01-27T12:09:31+01:00.vol1.difftar.gpg] Incsetlist: {} Start Time: 1106827771 End Time: 1106827771 PASSPHRASE variable not set, asking user. GnuPG passphrase: Running 'scp myuser@myhost:/usr/home/myuser/etc/duplicity-full.2005-01-27T12:09:31+01:00.manifest.gpg /tmp/tmpNajxHm' duplicity-full.2005-01-27T12:09:31+01:00.manifest.gpg 100% 735 0.7KB/s 00:00 gpg: encrypted with 2048-bit ELG-E key, ID 8B6B3F38, created 2005-01-26 "backup" Deleting /tmp/tmpNajxHm Selecting /home/pjs/a Comparing () and None Getting delta of (() /home/pjs/a dir) and None Generating delta - new file: . Selecting /home/pjs/a/c Comparing ('c',) and None Getting delta of (('c',) /home/pjs/a/c reg) and None Generating delta - new file: c Running 'scp /tmp/tmpTot6PH myuser@myhost:/usr/home/myuser/etc/duplicity-full.2005-01-27T12:09:53+01:00.vol1.difftar.gpg' tmpTot6PH 100% 742 0.7KB/s 00:00 Deleting /tmp/tmpTot6PH Running 'scp /tmp/tmp9WtN3Q myuser@myhost:/usr/home/myuser/etc/duplicity-full.2005-01-27T12:09:53+01:00.manifest.gpg' tmp9WtN3Q 100% 735 0.7KB/s 00:00 Deleting /tmp/tmp9WtN3Q Running 'scp /tmp/tmpv5zijV myuser@myhost:/usr/home/myuser/etc/duplicity-full-signatures.2005-01-27T12:09:53+01:00.sigtar.gpg' tmpv5zijV 100% 766 0.8KB/s 00:00 Deleting /tmp/tmpv5zijV Reading results of 'ssh myuser@myhost ls /usr/home/myuser/etc' Ignoring file 'duplicity-full-signatures.2005-01-27T12:09:31+01:00.sigtar.gpg' Ignoring file 'duplicity-full-signatures.2005-01-27T12:09:53+01:00.sigtar.gpg' Running 'ssh myuser@myhost rm /usr/home/myuser/etc/duplicity-full-signatures.2005-01-27T12:09:31+01:00.sigtar.gpg' --------------[ Backup Statistics ]-------------- ------------------------------------------------- After this we are left with this on the remote server: -rw------- 1 myuser users 766 Jan 27 12:10 duplicity-full-signatures.2005-01-27T12:09:53+01:00.sigtar.gpg -rw------- 1 myuser users 735 Jan 27 12:09 duplicity-full.2005-01-27T12:09:31+01:00.manifest.gpg -rw------- 1 myuser users 741 Jan 27 12:09 duplicity-full.2005-01-27T12:09:31+01:00.vol1.difftar.gpg -rw------- 1 myuser users 735 Jan 27 12:10 duplicity-full.2005-01-27T12:09:53+01:00.manifest.gpg -rw------- 1 myuser users 742 Jan 27 12:09 duplicity-full.2005-01-27T12:09:53+01:00.vol1.difftar.gpg Has anyone else found this? Thanks, Peter Spikings. --------------enig70B07CA2FE2EC3C7E1D423EF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (MingW32) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFB+N96J1UraO6qLPQRAouGAJ95Nbb4lzUle14XBA8tLIkwLHiAZACcDy9y 1GzCSowwapV6SSWbkzsJoO8= =QBvD -----END PGP SIGNATURE----- --------------enig70B07CA2FE2EC3C7E1D423EF--