[Bug-tar] Incomplete archives with "--listed-incremental"

bug-tar
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug-tar] Incomplete archives with "--listed-incremental"

From:	Wolfgang Denk
Subject:	[Bug-tar] Incomplete archives with "--listed-incremental"
Date:	Mon, 02 May 2005 00:39:48 +0200
Hello,

it seems there is a bug with the selection of files  to  be  archived
when "--listed-incremental" (resp. "-g") is used...

Here is what I did:

        # df -h / /boot /home /home/wd/Mail /opt
        Filesystem            Size  Used Avail Use% Mounted on
        /dev/hda3              12G  8.5G  3.2G  73% /
        /dev/hda1             248M   33M  215M  14% /boot
        /dev/md0               16G  5.8G   10G  37% /home
        /dev/hda8              13G  6.6G  6.5G  51% /home/wd/Mail
        /dev/md1               16G  7.3G  8.6G  46% /opt

Note: sum "Used": 28.2G

        # cd /
        # tar --totals -b 120 --label=atlas-level-0 \
        > -g /home/wd/BACKUP/2005-05-01-atlas-level-0 \
        > --rsh-command=/usr/bin/ssh \
        > -Mlcf diddl:/dev/dat . boot home home/wd/Mail opt
        ...
        Total bytes written: 15125544960 (14GB, 753kB/s)

Oops - that's only half of the expected data!

[Note: the file  "/home/wd/BACKUP/2005-05-01-atlas-level-0"  did  not
exist when I started tar.]

Closer inspection shows that the contents of the directories (=  file
systems)  "/boot",  "/home"  and  "/opt" is missing completely in the
archive. "/home/wd/Mail" is there, though.


Here is a test case (under Linux):

# mke2fs -m0 /dev/ram0
# mkdir /tmp/foo
# mount /dev/ram0 /tmp/foo
# mke2fs -m0 /dev/ram1
# mkdir /tmp/foo/boot
# mount /dev/ram1 /tmp/foo/boot
# mke2fs -m0 /dev/ram2
# mkdir /tmp/foo/home 
# mount /dev/ram2 /tmp/foo/home
# mke2fs -m0 /dev/ram3
# mkdir /tmp/foo/home/wd /tmp/foo/home/wd/Mail
# mount /dev/ram3 /tmp/foo/home/wd/Mail
# mke2fs -m0 /dev/ram4
# mkdir /tmp/foo/opt
# mount /dev/ram4 /tmp/foo/opt
# mount | grep ram
/dev/ram0 on /tmp/foo type ext2 (rw)
/dev/ram1 on /tmp/foo/boot type ext2 (rw)
/dev/ram2 on /tmp/foo/home type ext2 (rw)
/dev/ram3 on /tmp/foo/home/wd/Mail type ext2 (rw)
/dev/ram4 on /tmp/foo/opt type ext2 (rw)
# cd /tmp/foo
# echo aaaaa >root-1
# echo bbbbbb >root-2
# echo ccccccc >boot/boot-1
# echo dddddddd >boot/boot-2
# echo eeeeeeeee >home/home-1
# echo ffffffffff >home/home-2
# echo ggggggggggg >home/wd/wd-1
# echo hhhhhhhhhhhh >home/wd/wd-2
# echo iiiiiiiiiiiii >home/wd/Mail/Mail-1
# echo jjjjjjjjjjjjjj >home/wd/Mail/Mail-2
# echo kkkkkkkkkkkkkkk >opt/opt-1
# echo llllllllllllllll >opt/opt-2
# find . -exec ls -ld {} \;
drwxr-xr-x    6 root     root         1024 May  2 00:28 .
drwx------    2 root     root        12288 May  2 00:23 ./lost+found
drwxr-xr-x    3 root     root         1024 May  2 00:28 ./boot
drwx------    2 root     root        12288 May  2 00:24 ./boot/lost+found
-rw-r--r--    1 root     root            8 May  2 00:28 ./boot/boot-1
-rw-r--r--    1 root     root            9 May  2 00:28 ./boot/boot-2
drwxr-xr-x    4 root     root         1024 May  2 00:28 ./home
drwx------    2 root     root        12288 May  2 00:25 ./home/lost+found
drwxr-xr-x    3 root     root         1024 May  2 00:28 ./home/wd
drwxr-xr-x    3 root     root         1024 May  2 00:28 ./home/wd/Mail
drwx------    2 root     root        12288 May  2 00:27 
./home/wd/Mail/lost+found
-rw-r--r--    1 root     root           14 May  2 00:28 ./home/wd/Mail/Mail-1
-rw-r--r--    1 root     root           15 May  2 00:28 ./home/wd/Mail/Mail-2
-rw-r--r--    1 root     root           12 May  2 00:28 ./home/wd/wd-1
-rw-r--r--    1 root     root           13 May  2 00:28 ./home/wd/wd-2
-rw-r--r--    1 root     root           10 May  2 00:28 ./home/home-1
-rw-r--r--    1 root     root           11 May  2 00:28 ./home/home-2
drwxr-xr-x    3 root     root         1024 May  2 00:28 ./opt
drwx------    2 root     root        12288 May  2 00:27 ./opt/lost+found
-rw-r--r--    1 root     root           16 May  2 00:28 ./opt/opt-1
-rw-r--r--    1 root     root           17 May  2 00:28 ./opt/opt-2
-rw-r--r--    1 root     root            6 May  2 00:28 ./root-1
-rw-r--r--    1 root     root            7 May  2 00:28 ./root-2
# rm -f /tmp/snapshot /tmp/tarball
# tar --version
tar (GNU tar) 1.13.25
...
# tar --listed-incremental=/tmp/snapshot --multi-volume --one-file-system \
> --create --verbose --file=/tmp/tarball . boot home home/wd/Mail opt
tar: ./lost+found: Directory is new
tar: ./boot: Directory is new
tar: ./home: Directory is new
tar: ./opt: Directory is new
tar: boot/lost+found: Directory is new
tar: home/lost+found: Directory is new
tar: home/wd: Directory is new
tar: home/wd/Mail: Directory is new
tar: opt/lost+found: Directory is new
./
boot/
home/
home/wd/Mail/
opt/
./boot/
./home/
./lost+found/
./opt/
boot/lost+found/
home/lost+found/
home/wd/
home/wd/Mail/
opt/lost+found/
./root-1
./root-2
boot/boot-1
boot/boot-2
home/home-1
home/home-2
opt/opt-1
opt/opt-2
home/wd/wd-1
home/wd/wd-2
# tar tvf /tmp/tarball
drwxr-xr-x root/root        46 2005-05-02 00:28:23 ./
drwxr-xr-x root/root        29 2005-05-02 00:28:27 boot/
drwxr-xr-x root/root        33 2005-05-02 00:28:30 home/
drwxr-xr-x root/root         1 2005-05-02 00:28:41 home/wd/Mail/
drwxr-xr-x root/root        27 2005-05-02 00:28:47 opt/
drwxr-xr-x root/root         1 2005-05-02 00:28:27 ./boot/
drwxr-xr-x root/root         1 2005-05-02 00:28:30 ./home/
drwx------ root/root         1 2005-05-02 00:23:50 ./lost+found/
drwxr-xr-x root/root         1 2005-05-02 00:28:47 ./opt/
drwx------ root/root         1 2005-05-02 00:24:44 boot/lost+found/
drwx------ root/root         1 2005-05-02 00:25:25 home/lost+found/
drwxr-xr-x root/root        19 2005-05-02 00:28:35 home/wd/
drwxr-xr-x root/root         1 2005-05-02 00:28:41 home/wd/Mail/
drwx------ root/root         1 2005-05-02 00:27:53 opt/lost+found/
-rw-r--r-- root/root         6 2005-05-02 00:28:21 ./root-1
-rw-r--r-- root/root         7 2005-05-02 00:28:23 ./root-2
-rw-r--r-- root/root         8 2005-05-02 00:28:25 boot/boot-1
-rw-r--r-- root/root         9 2005-05-02 00:28:27 boot/boot-2
-rw-r--r-- root/root        10 2005-05-02 00:28:28 home/home-1
-rw-r--r-- root/root        11 2005-05-02 00:28:30 home/home-2
-rw-r--r-- root/root        16 2005-05-02 00:28:44 opt/opt-1
-rw-r--r-- root/root        17 2005-05-02 00:28:47 opt/opt-2
-rw-r--r-- root/root        12 2005-05-02 00:28:32 home/wd/wd-1
-rw-r--r-- root/root        13 2005-05-02 00:28:35 home/wd/wd-2

Here the contents of "home/wd/Mail"  is  missing  (even  though  this
directory is explicitely listed on the command line).

The results reported above were obtained using "tar (GNU tar) 1.13.25"
as distributed with RedHat Linux release 8.0.


If I build tar from the current sources the results are a little  bit
different,  but  the only difference is the order of the files in the
archive:

# tar --version
tar (GNU tar) 1.15.1
# tar --listed-incremental=/tmp/snapshot --multi-volume --one-file-system \
> --create --verbose --file=/tmp/tarball . boot home home/wd/Mail opt
tar: ./lost+found: Directory is new
tar: ./boot: Directory is new
tar: ./home: Directory is new
tar: ./opt: Directory is new
tar: boot/lost+found: Directory is new
tar: home/lost+found: Directory is new
tar: home/wd: Directory is new
tar: home/wd/Mail: Directory is new
tar: opt/lost+found: Directory is new
./
boot/
home/
home/wd/Mail/
opt/
./boot/
./home/
./lost+found/
./opt/
boot/lost+found/
home/lost+found/
home/wd/
home/wd/Mail/
opt/lost+found/
./root-1
./root-2
boot/boot-1
boot/boot-2
home/home-1
home/home-2
opt/opt-1
opt/opt-2
home/wd/wd-1
home/wd/wd-2
# tar tvf /tmp/tarball
drwxr-xr-x root/root        46 2005-05-02 00:28:23 ./
drwxr-xr-x root/root        29 2005-05-02 00:28:27 boot/
drwxr-xr-x root/root        33 2005-05-02 00:28:30 home/
drwxr-xr-x root/root         1 2005-05-02 00:28:41 home/wd/Mail/
drwxr-xr-x root/root        27 2005-05-02 00:28:47 opt/
drwxr-xr-x root/root         1 2005-05-02 00:28:27 ./boot/
drwxr-xr-x root/root         1 2005-05-02 00:28:30 ./home/
drwx------ root/root         1 2005-05-02 00:23:50 ./lost+found/
drwxr-xr-x root/root         1 2005-05-02 00:28:47 ./opt/
drwx------ root/root         1 2005-05-02 00:24:44 boot/lost+found/
drwx------ root/root         1 2005-05-02 00:25:25 home/lost+found/
drwxr-xr-x root/root        19 2005-05-02 00:28:35 home/wd/
drwxr-xr-x root/root         1 2005-05-02 00:28:41 home/wd/Mail/
drwx------ root/root         1 2005-05-02 00:27:53 opt/lost+found/
-rw-r--r-- root/root         6 2005-05-02 00:28:21 ./root-1
-rw-r--r-- root/root         7 2005-05-02 00:28:23 ./root-2
-rw-r--r-- root/root         8 2005-05-02 00:28:25 boot/boot-1
-rw-r--r-- root/root         9 2005-05-02 00:28:27 boot/boot-2
-rw-r--r-- root/root        10 2005-05-02 00:28:28 home/home-1
-rw-r--r-- root/root        11 2005-05-02 00:28:30 home/home-2
-rw-r--r-- root/root        16 2005-05-02 00:28:44 opt/opt-1
-rw-r--r-- root/root        17 2005-05-02 00:28:47 opt/opt-2
-rw-r--r-- root/root        12 2005-05-02 00:28:32 home/wd/wd-1
-rw-r--r-- root/root        13 2005-05-02 00:28:35 home/wd/wd-2
# ls -l home/wd/Mail
total 14
-rw-r--r--    1 root     root           14 May  2 00:28 Mail-1
-rw-r--r--    1 root     root           15 May  2 00:28 Mail-2
drwx------    2 root     root        12288 May  2 00:27 lost+found

Again, the contents of "home/wd/Mail" is missing.


Results differ if I use absolute path names:

# rm -f /tmp/snapshot /tmp/tarball
# tar --listed-incremental=/tmp/snapshot --multi-volume --one-file-system \
> --create --verbose --file=/tmp/tarball \
> /tmp/foo /tmp/foo/boot /tmp/foo/home /tmp/foo/home/wd/Mail /tmp/foo/opt
tar: /tmp/foo/lost+found: Directory is new
tar: /tmp/foo/boot: Directory is new
tar: /tmp/foo/home: Directory is new
tar: /tmp/foo/opt: Directory is new
tar: /tmp/foo/home/wd/Mail/lost+found: Directory is new
tar: Removing leading `/' from member names
/tmp/foo/
/tmp/foo/boot/
/tmp/foo/home/
/tmp/foo/home/wd/Mail/
/tmp/foo/opt/
/tmp/foo/boot/
/tmp/foo/home/
/tmp/foo/home/wd/Mail/lost+found/
/tmp/foo/lost+found/
/tmp/foo/opt/
/tmp/foo/root-1
/tmp/foo/root-2
/tmp/foo/home/wd/Mail/Mail-1
/tmp/foo/home/wd/Mail/Mail-2
# tar tvf /tmp/tarball
drwxr-xr-x root/root        46 2005-05-02 00:28:23 tmp/foo/
drwxr-xr-x root/root         1 2005-05-02 00:28:27 tmp/foo/boot/
drwxr-xr-x root/root         1 2005-05-02 00:28:30 tmp/foo/home/
drwxr-xr-x root/root        29 2005-05-02 00:28:41 tmp/foo/home/wd/Mail/
drwxr-xr-x root/root         1 2005-05-02 00:28:47 tmp/foo/opt/
drwxr-xr-x root/root         1 2005-05-02 00:28:27 tmp/foo/boot/
drwxr-xr-x root/root         1 2005-05-02 00:28:30 tmp/foo/home/
drwx------ root/root         1 2005-05-02 00:27:20 
tmp/foo/home/wd/Mail/lost+found/
drwx------ root/root         1 2005-05-02 00:23:50 tmp/foo/lost+found/
drwxr-xr-x root/root         1 2005-05-02 00:28:47 tmp/foo/opt/
-rw-r--r-- root/root         6 2005-05-02 00:28:21 tmp/foo/root-1
-rw-r--r-- root/root         7 2005-05-02 00:28:23 tmp/foo/root-2
-rw-r--r-- root/root        14 2005-05-02 00:28:39 tmp/foo/home/wd/Mail/Mail-1
-rw-r--r-- root/root        15 2005-05-02 00:28:41 tmp/foo/home/wd/Mail/Mail-2

Now all files in "boot", "home", "home/wd", and in "opt" are missing,
but "home/wd/Mail" now gets archived.



Best regards,

Wolfgang Denk

-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: address@hidden
I don't know if it's what you want, but it's what you get.  :-)
                      - Larry Wall in <address@hidden>
[Prev in Thread]
Current Thread
[Next in Thread]
[Bug-tar] Incomplete archives with "--listed-incremental", Wolfgang Denk, 2005/05/05
- Re: [Bug-tar] Incomplete archives with "--listed-incremental", Wolfgang Denk, 2005/05/06
- [Bug-tar] Incomplete archives with "--listed-incremental", Wolfgang Denk <=
Prev by Date: [Bug-tar] keep newer files bug?
Next by Date: Re: [Bug-tar] Incomplete archives with "--listed-incremental"
Previous by thread: Re: [Bug-tar] Incomplete archives with "--listed-incremental"
Next by thread: [Bug-tar] keep newer files bug?
Index(es):
- Date
- Thread