bug#13588: Pax hangs in case big UID

[Pavel Raiskup]

Hi, I'm just thinking aloud here.. I'm ok with this.

> Marc Herbert <address@hidden> adds (in bug#8343):
>
>     When "configure" is run by a user with an UID bigger than 21 bits,
>     BSD pax 3.4 aborts when trying to create the 'conftest.tar' test
>     archive and leaves an empty or corrupted conftest.tar file behind.
>     In the next step, pax tries to extract this incomplete or corrupted
>     archive and this *** hangs the whole ./configure script ***.
>
>     Note: GNU cpio 2.9 pretends to pass the test but it is a LIE: it
>     silently truncates any big UID to its lower 21 bits. I don't know
>     what can be the consequences of this lie.

I'm reading this once again and thinking about it  - and I was bad with
saying that 'ustar' may not success (it will not success completely).
This probably depends what we am__tar need for.., or?  Because the
resulted Makefile imo uses 'am__tar' just for creation distribution
tarball, we don't care about UID truncation.  This unnecessary UID/GID
information is not going to be used on user's (meaning user of the
tarball) machine.
...
So there could be probably considered the cpio good enough and better than
nothing?  But what if this is a bug in GNU cpio (and in others)?

I have done a little bit more observing here and found this a little even
more interesting.  Consider running everything under user with big UID:

  When 'star' is asked for 'ustar' format, it does not complain and works.
  When 'bsdtar' is asked for 'ustar' format, it complains, exits with 0
  return value and the resulted tarball is empty...  So..

  Another one - what if GNU tar behaviour is bad and the rule should be
  relaxed in future -> just warn user that the 'UID' is too big and
  truncate?

Definitely hard to detect.

-------

Ok, I think that we should not care about it *so* much and simply say that
it is impossible to store _correctly_ into 'ustar' (which will always be
truth) and set am__tar/am__untar to false as you propose.  Still, this
make-dist targets are used usually on up2date machines.  Usually not
running under big UIDs — and in that case, GNU tar should win.  At least,
upstream maintainers will always be able to use lower UID before
distribution.  At the end — people dealing with this problem are upstream
so they may easily switch to 'pax'.

The most important is to not hang during ./configure phase, and stop doing
it asap.

One thing as a future enhancement would be to allow redefine the am__tar
and am__untar by hand.  But it may be done later.

So *ACKing* :) with one simple remark which follows.

> [....]
> +    # The $UID and $GID variables are not portable, so we need to resort
> +    # to the POSIX-mandated id(1) utility.  Errors in the 'id' calls below
> +    # are definitely unexpected, so allow the users to see them (that is,
> +    # avoid stderr redirection).
> +    am_uid=`id -u || echo unknown`
> +    am_gid=`id -g || echo unknown`
> +    AC_MSG_CHECKING([whether UID '$am_uid' is supported by ustar format])
> +    if test $am_uid -le $am_max_uid; then
> +       AC_MSG_RESULT([yes])
> +    else
> +       AC_MSG_RESULT([no])
> +       _am_tools=none
> +    fi
> +    AC_MSG_CHECKING([whether GID '$am_gid' is supported by ustar format])
> +    if test $am_gid -le $am_max_gid; then
> +       AC_MSG_RESULT([yes])
> +    else
> +       AC_MSG_RESULT([no])
> +       _am_tools=none
> +    fi],

Here ^^ it would be possible to de-duplicate (just one message to standard
output and little simpler code).  But it is just a matter of taste.  To make
it easier, follows the patch, do not assign me as a co-author.

I'm also quite afraid of testsuite performance of testsuite - this
check costs more than 6 seconds..

Thanks a lot, Pavel

===========================================================================

commit 1f28bce58b3d98fcae26af3f848ecf900b2d43d1 (HEAD, branch-1.13.2)
Author:     Pavel Raiskup <address@hidden>
AuthorDate: Thu Apr 25 15:22:38 2013 +0200
Commit:     Pavel Raiskup <address@hidden>
CommitDate: Fri Apr 26 12:39:30 2013 +0200

    possible fixes

diff --git a/m4/tar.m4 b/m4/tar.m4
index 4b728e6..728afb4 100644
--- a/m4/tar.m4
+++ b/m4/tar.m4
@@ -36,23 +36,17 @@ m4_case([$1],
     [# Automake bugs #8343 and #13588: the ustar format doesn't support
     # an UID or a GID that require more than 21 bits to be stored.
     # In fact, the 'pax' utility can hang when such IDs are involved.
-    am_max_uid=2097151 # 2^21 - 1
-    am_max_gid=$am_max_uid
+    am_max_id=2097151 # 2^21 - 1
+
     # The $UID and $GID variables are not portable, so we need to resort
     # to the POSIX-mandated id(1) utility.  Errors in the 'id' calls below
     # are definitely unexpected, so allow the users to see them (that is,
     # avoid stderr redirection).
     am_uid=`id -u || echo unknown`
     am_gid=`id -g || echo unknown`
-    AC_MSG_CHECKING([whether UID '$am_uid' is supported by ustar format])
-    if test $am_uid -le $am_max_uid; then
-       AC_MSG_RESULT([yes])
-    else
-       AC_MSG_RESULT([no])
-       _am_tools=none
-    fi
-    AC_MSG_CHECKING([whether GID '$am_gid' is supported by ustar format])
-    if test $am_gid -le $am_max_gid; then
+
+    AC_MSG_CHECKING([whether UID and GID are storable in ustar archive])
+    if test $am_uid -le $am_max_id && test $am_gid -le $am_max_id; then
        AC_MSG_RESULT([yes])
     else
        AC_MSG_RESULT([no])
diff --git a/t/tar-ustar-id-too-high.sh b/t/tar-ustar-id-too-high.sh
index 79ae89d..441e8c2 100644
--- a/t/tar-ustar-id-too-high.sh
+++ b/t/tar-ustar-id-too-high.sh
@@ -62,8 +62,7 @@ PATH=$(pwd)/bin$PATH_SEPARATOR$PATH
 am_uid=16777216; export am_uid
 am_gid=1000;     export am_gid
 run_configure
-checked "whether UID '$am_uid' is supported by ustar format" "no"
-checked "whether GID '1000' is supported by ustar format" "yes"
+checked "whether UID and GID are storable in ustar archive" "no"
 checked "how to create a ustar tar archive" "none"
 
 # Another problematic ID reported in
@@ -71,8 +70,7 @@ checked "how to create a ustar tar archive" "none"
 am_uid=1000;     export am_uid
 am_gid=17000000; export am_gid
 run_configure
-checked "whether UID '1000' is supported by ustar format" "yes"
-checked "whether GID '$am_gid' is supported by ustar format" "no"
+checked "whether UID and GID are storable in ustar archive" "no"
 checked "how to create a ustar tar archive" "none"
 
 # The minimal ID that is too big.
@@ -81,8 +79,14 @@ two_to_twentyone=$((32 * 32 * 32 * 32 * 2))
 am_uid=$two_to_twentyone; export am_uid
 am_gid=$two_to_twentyone; export am_gid
 run_configure
-checked "whether UID '$two_to_twentyone' is supported by ustar format" "no"
-checked "whether GID '$two_to_twentyone' is supported by ustar format" "no"
+checked "whether UID and GID are storable in ustar archive" "no"
 checked "how to create a ustar tar archive" "none"
 
+# The maximal ID that is OK.
+two_to_twentyone_minus_one=$((32 * 32 * 32 * 32 * 2 - 1 ))
+# <https://bugzilla.redhat.com/show_bug.cgi?id=843376>.
+am_uid=$two_to_twentyone_minus_one; export am_uid
+am_gid=$two_to_twentyone_minus_one; export am_gid
+run_configure
+checked "whether UID and GID are storable in ustar archive" "yes"
 :