Re: [Bug-tar] keep newer files bug?

[Helmut Waitzmann]

Andrea Pellizzon <address@hidden> writes:

>I'm trying to use the option "--keep-newer files" with tar 1.15.1 and
>previous.

[...]

>$ tar xvf test.tar --keep-newer-files
>test
>tar: test: Warning: Cannot stat: No such file or directory
>tar: Current `test' is newer

The same goes here, as can be seen with the following listing (line
numbers prepended for easier reference).

  1 $ uname -srm
  2 Linux 2.6.8-2-686 i686
  3 $ tar --version
  4 tar (GNU tar) 1.14
  5 Copyright (C) 2004 Free Software Foundation, Inc.
  6 This program comes with NO WARRANTY, to the extent permitted by law.
  7 You may redistribute it under the terms of the GNU General Public License;
  8 see the file named COPYING for details.
  9 Written by John Gilmore and Jay Fenlason.
 10 $ rm -rf -- archive 1 2
 11 # Creating hierarchy    1/older/keep-newer-files
 12 # Hardlinking hierarchy 1/older/keep-newer-files
 13 # Creating hierarchy    1/older/keep-newer-files_overwrite
 14 # Hardlinking hierarchy 1/older/keep-newer-files_overwrite
 15 # Creating hierarchy    1/older/overwrite_keep-newer-files
 16 # Hardlinking hierarchy 1/older/overwrite_keep-newer-files
 17 # Creating hierarchy    1/older/keep-newer-files_unlink-first
 18 # Hardlinking hierarchy 1/older/keep-newer-files_unlink-first
 19 # Creating hierarchy    1/older/unlink-first_keep-newer-files
 20 # Hardlinking hierarchy 1/older/unlink-first_keep-newer-files
 21 $ sleep 3
 22 # Creating hierarchy    archive
 23 $ tar -c -f archive.tar -- -C archive file
 24 # table of contents of archive archive.tar:
 25 $ tar -tv -f archive.tar
 26 -rw------- helmut/helmut     5 2005-05-23 23:41:39 file
 27 $ sleep 3
 28 # Creating hierarchy    1/newer/keep-newer-files
 29 # Hardlinking hierarchy 1/newer/keep-newer-files
 30 # Creating hierarchy    1/newer/keep-newer-files_overwrite
 31 # Hardlinking hierarchy 1/newer/keep-newer-files_overwrite
 32 # Creating hierarchy    1/newer/overwrite_keep-newer-files
 33 # Hardlinking hierarchy 1/newer/overwrite_keep-newer-files
 34 # Creating hierarchy    1/newer/keep-newer-files_unlink-first
 35 # Hardlinking hierarchy 1/newer/keep-newer-files_unlink-first
 36 # Creating hierarchy    1/newer/unlink-first_keep-newer-files
 37 # Hardlinking hierarchy 1/newer/unlink-first_keep-newer-files
 38 $ sleep 3
 39 $ ls -dogi -ctr -- \
 40      1/older/keep-newer-files/file \
 41      2/older/keep-newer-files/file \
 42      1/older/keep-newer-files_overwrite/file \
 43      2/older/keep-newer-files_overwrite/file \
 44      1/older/overwrite_keep-newer-files/file \
 45      2/older/overwrite_keep-newer-files/file \
 46      1/older/keep-newer-files_unlink-first/file \
 47      2/older/keep-newer-files_unlink-first/file \
 48      1/older/unlink-first_keep-newer-files/file \
 49      2/older/unlink-first_keep-newer-files/file \
 50      archive/file \
 51      1/newer/keep-newer-files/file \
 52      2/newer/keep-newer-files/file \
 53      1/newer/keep-newer-files_overwrite/file \
 54      2/newer/keep-newer-files_overwrite/file \
 55      1/newer/overwrite_keep-newer-files/file \
 56      2/newer/overwrite_keep-newer-files/file \
 57      1/newer/keep-newer-files_unlink-first/file \
 58      2/newer/keep-newer-files_unlink-first/file \
 59      1/newer/unlink-first_keep-newer-files/file \
 60      2/newer/unlink-first_keep-newer-files/file
 61 831408 -rw-------  2  0 2005-05-23 23:41:35+0200 
2/older/keep-newer-files/file
 62 831408 -rw-------  2  0 2005-05-23 23:41:35+0200 
1/older/keep-newer-files/file
 63 831413 -rw-------  2  1 2005-05-23 23:41:35+0200 
2/older/keep-newer-files_overwrite/file
 64 831413 -rw-------  2  1 2005-05-23 23:41:35+0200 
1/older/keep-newer-files_overwrite/file
 65 831416 -rw-------  2  2 2005-05-23 23:41:35+0200 
2/older/overwrite_keep-newer-files/file
 66 831416 -rw-------  2  2 2005-05-23 23:41:35+0200 
1/older/overwrite_keep-newer-files/file
 67 831419 -rw-------  2  3 2005-05-23 23:41:36+0200 
2/older/keep-newer-files_unlink-first/file
 68 831419 -rw-------  2  3 2005-05-23 23:41:36+0200 
1/older/keep-newer-files_unlink-first/file
 69 831422 -rw-------  2  4 2005-05-23 23:41:36+0200 
2/older/unlink-first_keep-newer-files/file
 70 831422 -rw-------  2  4 2005-05-23 23:41:36+0200 
1/older/unlink-first_keep-newer-files/file
 71 831425 -rw-------  1  5 2005-05-23 23:41:39+0200 archive/file
 72 831429 -rw-------  2  6 2005-05-23 23:41:42+0200 
2/newer/keep-newer-files/file
 73 831429 -rw-------  2  6 2005-05-23 23:41:42+0200 
1/newer/keep-newer-files/file
 74 831433 -rw-------  2  7 2005-05-23 23:41:42+0200 
2/newer/keep-newer-files_overwrite/file
 75 831433 -rw-------  2  7 2005-05-23 23:41:42+0200 
1/newer/keep-newer-files_overwrite/file
 76 831436 -rw-------  2  8 2005-05-23 23:41:42+0200 
2/newer/overwrite_keep-newer-files/file
 77 831436 -rw-------  2  8 2005-05-23 23:41:42+0200 
1/newer/overwrite_keep-newer-files/file
 78 831439 -rw-------  2  9 2005-05-23 23:41:42+0200 
2/newer/keep-newer-files_unlink-first/file
 79 831439 -rw-------  2  9 2005-05-23 23:41:42+0200 
1/newer/keep-newer-files_unlink-first/file
 80 831442 -rw-------  2 10 2005-05-23 23:41:42+0200 
2/newer/unlink-first_keep-newer-files/file
 81 831442 -rw-------  2 10 2005-05-23 23:41:42+0200 
1/newer/unlink-first_keep-newer-files/file
 82 $ tar -x -f archive.tar --keep-newer-files \
 83       -- -C 1/older/keep-newer-files
 84 tar: file: Warning: Cannot stat: No such file or directory
 85 tar: Current `file' is newer
 86 $ ls -dogi -ctr -- 1/older/keep-newer-files/file
 87 ls: 1/older/keep-newer-files/file: No such file or directory
 88 $ tar -x -f archive.tar --keep-newer-files --overwrite \
 89       -- -C 1/older/keep-newer-files_overwrite
 90 $ ls -dogi -ctr -- 1/older/keep-newer-files_overwrite/file
 91 831413 -rw-------  2 5 2005-05-23 23:41:45+0200 
1/older/keep-newer-files_overwrite/file
 92 $ tar -x -f archive.tar --overwrite --keep-newer-files \
 93       -- -C 1/older/overwrite_keep-newer-files
 94 tar: file: Warning: Cannot stat: No such file or directory
 95 tar: Current `file' is newer
 96 $ ls -dogi -ctr -- 1/older/overwrite_keep-newer-files/file
 97 ls: 1/older/overwrite_keep-newer-files/file: No such file or directory
 98 $ tar -x -f archive.tar --keep-newer-files --unlink-first \
 99       -- -C 1/older/keep-newer-files_unlink-first
100 $ ls -dogi -ctr -- 1/older/keep-newer-files_unlink-first/file
101 831444 -rw-------  1 5 2005-05-23 23:41:45+0200 
1/older/keep-newer-files_unlink-first/file
102 $ tar -x -f archive.tar --unlink-first --keep-newer-files \
103       -- -C 1/older/unlink-first_keep-newer-files
104 tar: file: Warning: Cannot stat: No such file or directory
105 tar: Current `file' is newer
106 $ ls -dogi -ctr -- 1/older/unlink-first_keep-newer-files/file
107 ls: 1/older/unlink-first_keep-newer-files/file: No such file or directory
108 $ tar -x -f archive.tar --keep-newer-files \
109       -- -C 1/newer/keep-newer-files
110 tar: Current `file' is newer
111 $ ls -dogi -ctr -- 1/newer/keep-newer-files/file
112 831429 -rw-------  2 6 2005-05-23 23:41:42+0200 
1/newer/keep-newer-files/file
113 $ tar -x -f archive.tar --keep-newer-files --overwrite \
114       -- -C 1/newer/keep-newer-files_overwrite
115 $ ls -dogi -ctr -- 1/newer/keep-newer-files_overwrite/file
116 831433 -rw-------  2 5 2005-05-23 23:41:45+0200 
1/newer/keep-newer-files_overwrite/file
117 $ tar -x -f archive.tar --overwrite --keep-newer-files \
118       -- -C 1/newer/overwrite_keep-newer-files
119 tar: Current `file' is newer
120 $ ls -dogi -ctr -- 1/newer/overwrite_keep-newer-files/file
121 831436 -rw-------  2 8 2005-05-23 23:41:42+0200 
1/newer/overwrite_keep-newer-files/file
122 $ tar -x -f archive.tar --keep-newer-files --unlink-first \
123       -- -C 1/newer/keep-newer-files_unlink-first
124 $ ls -dogi -ctr -- 1/newer/keep-newer-files_unlink-first/file
125 831445 -rw-------  1 5 2005-05-23 23:41:45+0200 
1/newer/keep-newer-files_unlink-first/file
126 $ tar -x -f archive.tar --unlink-first --keep-newer-files \
127       -- -C 1/newer/unlink-first_keep-newer-files
128 tar: Current `file' is newer
129 $ ls -dogi -ctr -- 1/newer/unlink-first_keep-newer-files/file
130 831442 -rw-------  2 10 2005-05-23 23:41:42+0200 
1/newer/unlink-first_keep-newer-files/file

First (lines 11 to 20), five file hierarchies are built in the
subdirectory '1', each of them containing a file 'file' of unique length
(to be able to distinguish them when extracting).  These hierarchies are
recursively hardlinked to corresponding hierarchies in subdirectory '2'
in order to distinguish whether tar overwrites or unlinks when
extracting.

After waiting 3 seconds, the 'archive' hierarchy is created the same way
(but not hardlinked) and stored in the 'archive.tar' file.

After waiting 3 seconds, five file hierarchies are built in
the same way and hardlinked as before (lines 28 to 37).

All created files can be seen at lines 61 to 81.  The option "-c" is
supplied to the "ls" command to let it show the file status change time
rather than the data modification time (which could be reset by tar when
extracting archive members).

When extracting files, tar's option '--keep-newer-files' reveals some
errors:

First condition:  The file in the file system is OLDER than the archive
member to be extracted from the archive.

So tar should extract it.

(1) As can be seen in lines 82 to 87, tar does not do that:  It unlinks
    the file but fails to extract the archive member, which is an ERROR.

(2) If '--keep-newer-files' is FOLLOWED by the additional option
    '--overwrite', then the archive member is extracted (see lines 88 to
    91).  The file is not unlinked (its 2 hardlinks remain) but simply
    overwritten, which is CORRECT.

(3) The same error like in (1) arises, when option '--keep-newer-files' is
    PRECEDED by option '--overwrite' (see lines 92 to 97):  The old file
    is unlinked and no file is created.

(4) If '--keep-newer-files' is FOLLOWED by the additional option
    '--unlink-first', then the archive member is extracted (see line 98
    to 101).  The file is unlinked (the hardlink is broken), which is
    CORRECT.

(5) The same error like in (1) arises, when option '--keep-newer-files'
    is PRECEDED by option '--unlink-first' (see lines 102 to 107):  The
    old file is unlinked and no file is created.

Second condition:  The file in the file system is NEWER than the archive
member to be extracted from the archive.

So tar should not extract it.

(6) Tar does not extract the archive member (lines 108 to 112):  The file
    status change date, the link count, and the file size remain
    unchanged (compare with line 73), which is CORRECT.

(7) If '--keep-newer-files' is FOLLOWED by '--overwrite', then the
    archive member is extracted (see lines 113 to 116).  The file is
    overwritten rather than preserved.  The link count remains 2, but the
    file size and the file status change date is changed, which is an
    ERROR.

(8) If '--keep-newer-files' is PRECEDED by option '--overwrite' (lines
    117 to 121), the file is preserved like in (6), which is CORRECT.

(9) If '--keep-newer-files' is FOLLOWED by '--unlink-first', then the
    archive member is extracted (lines 122 to 125).  The file is unlinked
    and created rather than preserved.  The hardlink is broken, and the
    file size and the file status change date is changed, which is an
    ERROR.

(10) If '--keep-newer-files' is PRECEDED by option '--unlink-first (lines
     126 to 130), the file is preserved like in (6), which is CORRECT.

This is the shell script, which produced the listing above (pipe it to
"pr -tTi -n' 3'" to get line numbers):

my_test_gnu_tar
Description: shell script to test gnu tar's '--keep-newer-files' option

-- 
Wenn Sie mir E-Mail schreiben, stellen |  When writing me e-mail, please
Sie bitte vor meine E-Mail-Adresse     |  precede my e-mail address with
meinen Vor- und Nachnamen, etwa so:    |  my full name, like
Helmut Waitzmann <address@hidden>, (Helmut Waitzmann) address@hidden