Re: [Help-tar] What is the file size limit for tar?

[Jakob Bohm]

On 13/04/2015 16:54, Ian Bentley wrote:
> Hi all,
>
> My company is running into problems with tar, because we need a 
> container for quite large files.  There are varying reports to the 
> maximum file-size that tar can support, but it appears that it was 
> originally 2GB, later extended to 8GB, and most recently extended to 
> 68 GB.  Admitadly these numbers are based on non-GNU documentation, 
> because I can't find much in the way of documentation.
>
> Apache Compress Commons (Java) 's most recent version still caps the 
> filesize at 8GB[1].  Citing the max value represented by 11 Octal chars.
>
> Testing on a Mac OS X machine with tar (GNU tar) 1.27.1  with standard 
> options worked successfully on an 80GB file.
>
> Quickly inspecting the GNU Tar source this morning I was able to 
> follow the path from "xheader.c" `size_coder` => eventually "misc.c" 
> `sysinttostr` which implies that the file size is getting encoded as a 
> string.  Before digging more into the GNUTar codebase, I wondered if I 
> could get some clarification about the following questions.
>
> What is the largest file that a tar can store?
> Did I get "lucky" that my tar completed successfully on such a large file?
> Is the Apache Compress Commons implementation wrong?
When using the POSIX.1-2001 format aka pax, there is no
upper limit at all, this is requested by using the
-H posix option to GNU tar or by using the tar-program
named pax in some other POSIX systems.

When using either of the special GNU tar format, there
is no upper limit either, these are selected by using
the -H gnu or -H oldgnu options.

Either of those 3 formats are usually the default.

The 8GB limit applied to the historic formats
v7 (UNIX V7) and ustar (POSIX.1-1988).

For more information, see this page from the GNU tar
documentation (which should also be available locally
via the "info" command):

https://www.gnu.org/software/tar/manual/html_node/Formats.html

Maybe the commons-compress library only implements the old
1988 ustarformat.

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded