[rdiff-backup-users] Re: Memory problems - any suggestions?

[David]

On Mon, May 5, 2008 at 11:33 AM, David <address@hidden> wrote:
> Hi, rdiff-backup is using excessive amounts of memory on my system
>  (almost 2 GB used on my system with 1 GB RAM and 1 GB swap).
>

Some more info. I'm busy running rdiff-backup now (with that excessive
memory usage), with the -v9 option.

Here is the most recent output:

=====OUTPUT STARTS=====
Mon May  5 06:49:03 2008  Touching
/data/backups/rrbackups/uberbackup1_mirror/files/rdiff-backup-data/increments
/data/backups/rrbackups/complete_server_bkps/192.168.0.23/files/rdiff-backup-data/increments/var/www/dwww.2007-1
0-11T02:01:47+02:00.dir.2007-10-04T15:54:38+02:00.missing
Mon May  5 06:49:03 2008  Renaming
/data/backups/rrbackups/uberbackup1_mirror/files/data/backups/rrbackups/compl
ete_server_bkps/192.168.0.23/files/rdiff-backup-data/increments/var/www/rdiff-backup.tmp.538914
to /data/backups
/rrbackups/uberbackup1_mirror/files/data/backups/rrbackups/complete_server_bkps/192.168.0.23/files/rdiff-backup-
data/increments/var/www/dwww.2007-10-11T02:01:47+02:00.dir
Mon May  5 06:49:07 2008  Processing changed file
data/backups/rrbackups/complete_server_bkps/192.168.0.23/files
/rdiff-backup-data/increments/var/www/mp3join.2007-10-09T01:47:48+02:00.dir
Mon May  5 06:49:07 2008  Regular copying ('data', 'backups',
'rrbackups', 'complete_server_bkps', '192.168.0.23 ', 'files',
'rdiff-backup-data', 'increments', 'var', 'www',
'mp3join.2007-10-09T01:47:48+02:00.dir') to /data/b
ackups/rrbackups/uberbackup1_mirror/files/data/backups/rrbackups/complete_server_bkps/192.168.0.23/files/rdiff-b
ackup-data/increments/var/www/rdiff-backup.tmp.538915
Mon May  5 06:49:07 2008  Writing file object to
/data/backups/rrbackups/uberbackup1_mirror/files/data/backups/r
rbackups/complete_server_bkps/192.168.0.23/files/rdiff-backup-data/increments/var/www/rdiff-backup.tmp.538915
Mon May  5 06:49:07 2008  Copying attributes from ('data', 'backups',
'rrbackups', 'complete_server_bkps', '192. 168.0.23', 'files',
'rdiff-backup-data', 'increments', 'var', 'www',
'mp3join.2007-10-09T01:47:48+02:00.dir') to
/data/backups/rrbackups/uberbackup1_mirror/files/data/backups/rrbackups/complete_server_bkps/192.168.0.23/files
/rdiff-backup-data/increments/var/www/rdiff-backup.tmp.538915
Mon May  5 06:49:07 2008  Setting time of
/data/backups/rrbackups/uberbackup1_mirror/files/data/backups/rrbackup
s/complete_server_bkps/192.168.0.23/files/rdiff-backup-data/increments/var/www/rdiff-backup.tmp.538915
to 118577 9828
Mon May  5 06:49:07 2008  Incrementing mirror file
/data/backups/rrbackups/uberbackup1_mirror/files/data/backups
/rrbackups/complete_server_bkps/192.168.0.23/files/rdiff-backup-data/increments/var/www/mp3join.2007-10-09T01:47
:48+02:00.dir
Mon May  5 06:49:07 2008  Touching
/data/backups/rrbackups/uberbackup1_mirror/files/rdiff-backup-data/increments
/data/backups/rrbackups/complete_server_bkps/192.168.0.23/files/rdiff-backup-data/increments/var/www/mp3join.200
7-10-09T01:47:48+02:00.dir.2007-10-04T15:54:38+02:00.missing
Mon May  5 06:49:07 2008  Renaming
/data/backups/rrbackups/uberbackup1_mirror/files/data/backups/rrbackups/compl
ete_server_bkps/192.168.0.23/files/rdiff-backup-data/increments/var/www/rdiff-backup.tmp.538915
to /data/backups
/rrbackups/uberbackup1_mirror/files/data/backups/rrbackups/complete_server_bkps/192.168.0.23/files/rdiff-backup-
data/increments/var/www/mp3join.2007-10-09T01:47:48+02:00.dir

=====OUTPUT ENDS=====

(These are the real directory paths, unlike my simplified directory
names in my previous examples).

It's been running that last bit for the past 6 hours.

The "rdiff-backup.tmp.538915" directory (currently, rdiff-backup is
still busy with the dir) has 252220  files and 12744  directories, for
a total of 83M (looks like it's mostly ".missing" rdiff-backup
markers). Memory usage is as mentioned before.

Should I expect rdiff-backup's 'rename' operation to perform this
badly? (I'm more concerned about the memory usage than the time it
takes).

David.