[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[rdiff-backup-users] Failing on 11GB file - other large files working
From: |
Ty! Boyack |
Subject: |
[rdiff-backup-users] Failing on 11GB file - other large files working |
Date: |
Fri, 18 Jul 2003 12:10:26 -0600 |
We've been doing nightly backups for a couple of weeks now without any
problems, and then a user throws another kink at us. I've attached a
trace below of the entire backup run, but the file it is failing on is
an 11GB file, of which about 5GB shows up on the backup mirror.
We regularly handle files larger than 2GB, so I thought we were done
with the large file problems, and we run with --no-compression as we
still have Python 2.2.1 and are limited to 2GB zip files if we don't
disable compression.
I'm wondering if there may still be a 32 bit offset index that is used
when building the delta. Granted that I know very little about the
inner workings of rdiff-backup (or librsync), so that's just a guess.
Any ideas on how to get large files like this (this file will have
very large block-level differences) to get backed up successfully?
Thanks,
-Ty!
(rdiff-backup 0.11.4 + patches from Ben, libsync 0.9.5.1, Python
2.2.1, Linux Redhat 8.0 w/ kernel 2.4.21)
------------------------------------------------------------------
Command:
rdiff-backup -v9 --no-compression /export/nrel-disk01-vga/hobbs.main
/export/nrel-disk01-vga/BACKUP/hobbs.main
Output:
Thu Jul 17 11:22:46 2003 Touching mirror marker
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/current_mirror.2003-07-17T11:22:46-06:00.data
Thu Jul 17 11:22:46 2003 Touching
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/current_mirror.2003-07-17T11:22:46-06:00.data
Thu Jul 17 11:22:53 2003 Getting signature of hobbs
backup/backup2.bkf
Thu Jul 17 11:22:53 2003 Getting delta of
/export/nrel-disk01-vga/hobbs.main/hobbs backup/backup2.bkf with
signature hobbs backup/backup2.bkf
Thu Jul 17 11:25:42 2003 Processing changed file .
Thu Jul 17 11:25:42 2003 Incrementing mirror file
/export/nrel-disk01-vga/BACKUP/hobbs.main
Thu Jul 17 11:25:42 2003 Touching
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/increments.2003-07-17T10:16:22-06:00.dir
Thu Jul 17 11:25:42 2003 Copying attributes from () to
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/increments.2003-07-17T10:16:22-06:00.dir
Thu Jul 17 11:25:42 2003 Setting time of
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/increments.2003-07-17T10:16:22-06:00.dir
to 1058454062
Thu Jul 17 11:25:42 2003 Processing changed file hobbs backup
Thu Jul 17 11:25:42 2003 Incrementing mirror file
/export/nrel-disk01-vga/BACKUP/hobbs.main/hobbs backup
Thu Jul 17 11:25:42 2003 Touching
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/increments/hobbs
backup.2003-07-17T10:16:22-06:00.dir
Thu Jul 17 11:25:42 2003 Copying attributes from ('hobbs backup',) to
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/increments/hobbs
backup.2003-07-17T10:16:22-06:00.dir
Thu Jul 17 11:25:42 2003 Setting time of
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/increments/hobbs
backup.2003-07-17T10:16:22-06:00.dir to 1050711261
Thu Jul 17 11:25:42 2003 Processing changed file hobbs
backup/backup2.bkf
Thu Jul 17 11:25:42 2003 Writing file object to
/export/nrel-disk01-vga/BACKUP/hobbs.main/hobbs
backup/rdiff-backup.tmp.0
python: ERROR: (rs_file_copy_cb) seek failed: Invalid argument
python: ERROR: (rs_job_complete) patch job failed: IO error
Thu Jul 17 11:30:29 2003 Exception 'librsync error 100 while in patch
cycle' raised of class 'rdiff_backup.librsync.librsyncError':
File
"/usr/local/rdiff-backup-0.11.4/lib/python2.2/site-packages/rdiff_backup/robust.py",
line 32, in check_common_error
try: return function(*args)
File
"/usr/local/rdiff-backup-0.11.4/lib/python2.2/site-packages/rdiff_backup/Rdiff.py",
line 76, in patch_local
if outrp: outrp.write_from_fileobj(patchfile)
File
"/usr/local/rdiff-backup-0.11.4/lib/python2.2/site-packages/rdiff_backup/rpath.py",
line 828, in write_from_fileobj
copyfileobj(fp, outfp)
File
"/usr/local/rdiff-backup-0.11.4/lib/python2.2/site-packages/rdiff_backup/rpath.py",
line 58, in copyfileobj
inbuf = inputfp.read(blocksize)
File
"/usr/local/rdiff-backup-0.11.4/lib/python2.2/site-packages/rdiff_backup/librsync.py",
line 76, in read
self._add_to_outbuf_once()
File
"/usr/local/rdiff-backup-0.11.4/lib/python2.2/site-packages/rdiff_backup/librsync.py",
line 87, in _add_to_outbuf_once
except _librsync.librsyncError, e: raise librsyncError(str(e))
Thu Jul 17 11:30:29 2003 UpdateError hobbs backup/backup2.bkf
librsync error 100 while in patch cycle
Thu Jul 17 11:30:30 2003 Deleting
/export/nrel-disk01-vga/BACKUP/hobbs.main/hobbs
backup/rdiff-backup.tmp.0
Thu Jul 17 11:37:12 2003 Copying attributes from ('hobbs backup',) to
/export/nrel-disk01-vga/BACKUP/hobbs.main/hobbs backup
Thu Jul 17 11:37:12 2003 Setting time of
/export/nrel-disk01-vga/BACKUP/hobbs.main/hobbs backup to 1050711261
Thu Jul 17 11:37:12 2003 Copying attributes from () to
/export/nrel-disk01-vga/BACKUP/hobbs.main
Thu Jul 17 11:37:12 2003 Setting time of
/export/nrel-disk01-vga/BACKUP/hobbs.main to 1058454062
Thu Jul 17 11:37:13 2003 Deleting
/export/nrel-disk01-vga/BACKUP/hobbs.main/rdiff-backup-data/current_mirror.2003-07-17T10:16:22-06:00.data
Thu Jul 17 11:37:13 2003 Cleaning up
--
-===========================-
Ty! Boyack
NREL Unix Network Manager
address@hidden
(970) 491-1186
-===========================-
- [rdiff-backup-users] Failing on 11GB file - other large files working,
Ty! Boyack <=