Re: [Bug-tar] Problem with gnu tar and ZFS on Linux

[Joerg Schilling]

Paul Eggert <address@hidden> wrote:

> The problem with this particular filesystem appears to be that it lies 
> about inode numbers in its snapshots. I suppose GNU tar (and other 
> programs) will have to add a flag to ignore inode numbers too. What a pain.

In case that all inode numbers are different, this filesysem is completely 
unusable. If only the inode number for the root directory is wrong, I cannot 
say without testing, whether there is a way to make star deal with that problem
and I cannot speak for gtar.

This implementation decision is definitely in conflict with the basic ZFS 
design rules that grant that inode numbers > 2147483647 will only be created 
in case that the file system already has 2147483647 files which results in a 
filesystem size of at least 8 TB and a filesystem that has been completely 
filled up with smallest files.


> By the way, would you mind toning down the ads in your emails? This 
> mailing list is for improving GNU Tar, not for advertising other 
> software. Thanks.

Well, the name of the mailing list is "bug-tar" and not "bug-gtar", so people 
would assume that it is a general tar list and not just a gtar specific list.

Since you asked, I should be more explicit and explain why I have been 
carefully hinting people to use different solutions:

Since 1990, gtar claims that it supports incremental backups and restores but 
it seems that incremental restores never have been tested with the needed care.

When star's inremental support was initially written in September 2004, I did 
run some basic tests in order to check whether star was on the right way and in 
order to compare star with other solutions. It turned out that gtar did not 
pass the basic tests and I made a bug report against that gtar problem.

The method introduced by ufsdump around 1981 and used as the basic concept for 
star however works and incrementals with star never caused any problem since 
February 2005 when star incrementals have been declared to be ready for use.

Now nearly 14 years after the gtar problems have been detected and reported,
the basic problem with gtar still exists and from a quick web search, it seems 
that I send a reminder at least in 2011 and 2016 already.

Let me give you a script to reproduce the problem:

/*--------------------------------------------------------------------------*/
if [ "$gtar" ]; then
        #
        # Permit: gtar=/tmp/tar-1.30/src/tar sh gnutarfail.sh
        #
        GT=`"$gtar" --help 2> /dev/null | grep GNU`
else
        GT=`gtar --help 2> /dev/null | grep GNU`
        if [ "$GT" ]; then
                gtar=gtar
        else
                # Some systems have "gtar" installed as "tar"
                GT=`tar --help 2> /dev/null | grep GNU`
                if [ "$GT" ]; then
                        gtar=tar
                fi
        fi
fi
if [ -z "$GT" ]; then
        echo No gtar found
        exit 1
fi
echo gtar installed as $gtar
# Preparation complete
#-----------------------------------

cd /tmp
mkdir test.$$
cd test.$$

set -x

mkdir test
mkdir test/dir1
mkdir test/dir2

echo dir1-file > test/dir1/dir1-file
echo dir2-file > test/dir2/dir2-file

$gtar -g/tmp/test.$$/listed-incr -c -f /tmp/test.$$/full.tar test

rm -rf test/dir2
mv test/dir1 test/dir2

$gtar -g/tmp/test.$$/listed-incr -c -f /tmp/test.$$/incremental.tar test

mv test orig

$gtar -x -g/dev/null -f /tmp/test.$$/full.tar
$gtar -x -g/dev/null -f /tmp/test.$$/incremental.tar
/*--------------------------------------------------------------------------*/

If you run that script, you get:

LC_ALL=C sh gnutarfail.sh 
gtar installed as gtar
+ mkdir test 
+ mkdir test/dir1 
+ mkdir test/dir2 
+ echo dir1-file 
+ echo dir2-file 
+ gtar -g/tmp/test.6611/listed-incr -c -f /tmp/test.6611/full.tar test 
+ rm -rf test/dir2 
+ mv test/dir1 test/dir2 
+ gtar -g/tmp/test.6611/listed-incr -c -f /tmp/test.6611/incremental.tar test 
+ mv test orig 
+ gtar -x -g/dev/null -f /tmp/test.6611/full.tar 
+ gtar -x -g/dev/null -f /tmp/test.6611/incremental.tar 
gtar: Cannot rename `test/dir1' to `test/dir2': File exists
gtar: Error exit delayed from previous errors

I am not sure whether it is possible to solve the problem without introducing a 
new incompatible dump format in GNU tar. AFAIK Gnu tar tries to detect and 
understand all changes while creating the archive by using a partial database 
during create. Star detects and understands changes at extract time by using a 
complete database which coveres all files.

See e.g.:

https://lists.gnu.org/archive/html/bug-tar/2016-07/msg00026.html

Jörg

-- 
 EMail:address@hidden                    (home) Jörg Schilling D-13353 Berlin
    address@hidden (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'