file systems with 64-bit inodes

[Bruno Haible]

So far, we have documented that the AC_SYS_LARGEFILE macro (from module
'largefile') is useful for accessing files > 2 GiB.

Now, Dmitry Levin points out in [1] that this macro is also needed to
make stat() and fstat() "work properly on file systems with 64-bit inodes".

I think what he means is the "64 bit inode problem": stat(), lstat(),
fstat() fail with EOVERFLOW if
  - the Linux file system has 64-bit inodes enabled,
  - the file's inode happens to not fit in 32 bits,
  - the application is a 32-bit Linux/x86 application (possibly running
    on Linux/x86_64).

In fact, this is what I see in /usr/include/asm/stat.h on a Linux/x86_64
machine:

#ifdef __i386__
struct stat {
...
        unsigned long  st_ino; /* 32 bits */
...
};
struct stat64 {
...
        unsigned long  __st_ino; /* 32 bits */
...
        unsigned long long      st_ino; /* 64 bits */
...
};
#else
struct stat {
...
        __kernel_ulong_t        st_ino;
..
};
#endif

It confirms what was said in that article and that using stat64() instead
of stat() will fix the issue.

Which file systems are affected?
- Ext4 apparently not [3][4]
- XFS [5]
- NFS [6]

Are other OSes affected?
- FreeBSD is said to have achieved 64-bit inode support "while retaining
  backwards compatibility" - so, no AC_SYS_LARGEFILE is needed here. [7]

[1] https://lists.gnu.org/archive/html/bug-gettext/2020-01/msg00001.html
[2] https://www.mjr19.org.uk/sw/inodes64.html
[3] https://steamcommunity.com/app/221410/discussions/0/620695877288637183/
[4] https://www.kernel.org/doc/html/latest/filesystems/ext4/index.html
[5] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1361637
[6] 
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Release_Notes/U7/ppc/ar01s04.html
[7] 
https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-64-bit-Inodes-ino64


I'm therefore adding this doc update in gnulib. Does it sound correct?

The autoconf documentation of AC_SYS_LARGEFILE would also need to be updated.


2020-01-03  Bruno Haible  <address@hidden>

        doc: Mention the 64-bit inode number problem.
        * m4/largefile.m4 (AC_SYS_LARGEFILE): Mention that this macro fixes the
        64-bit inode number problem.
        * doc/posix-functions/stat.texi: Mention that this module fixes the
        64-bit inode number problem.
        * doc/posix-functions/lstat.texi: Likewise.
        * doc/posix-functions/fstat.texi: Likewise.
        * doc/posix-functions/readdir.texi: Add more details.
        * doc/posix-functions/readdir_r.texi: Likewise.

diff --git a/doc/posix-functions/fstat.texi b/doc/posix-functions/fstat.texi
index 579ea42..a404891 100644
--- a/doc/posix-functions/fstat.texi
+++ b/doc/posix-functions/fstat.texi
@@ -16,6 +16,11 @@ On platforms where @code{off_t} is a 32-bit type, 
@code{fstat} may not correctly
 report the size of files or block devices larger than 2 GB.
 (Cf. @code{AC_SYS_LARGEFILE}.)
 @item
+On Linux/x86 and Linux/x86_64, applications compiled in 32-bit mode cannot
+access files that happen to have a 64-bit inode number.  This can occur with
+file systems such as XFS (typically on large disks) and NFS.
+(Cf. @code{AC_SYS_LARGEFILE}.)
+@item
 On Solaris 11.4, when this function yields a timestamp with a
 nonpositive @code{tv_sec} value, @code{tv_nsec} might be in the range
 @minus{}1000000000..@minus{}1, representing a negative nanoseconds
diff --git a/doc/posix-functions/lstat.texi b/doc/posix-functions/lstat.texi
index 9a03cfb..1b1aff6 100644
--- a/doc/posix-functions/lstat.texi
+++ b/doc/posix-functions/lstat.texi
@@ -13,6 +13,11 @@ On platforms where @code{off_t} is a 32-bit type, 
@code{lstat} may not
 correctly report the size of files or block devices larger than 2 GB.
 (Cf. @code{AC_SYS_LARGEFILE}.)
 @item
+On Linux/x86 and Linux/x86_64, applications compiled in 32-bit mode cannot
+access files that happen to have a 64-bit inode number.  This can occur with
+file systems such as XFS (typically on large disks) and NFS.
+(Cf. @code{AC_SYS_LARGEFILE}.)
+@item
 For symlinks, when the argument ends in a slash, some platforms don't
 dereference the argument:
 Solaris 9.
diff --git a/doc/posix-functions/readdir.texi b/doc/posix-functions/readdir.texi
index 61ac883..3ec36bd 100644
--- a/doc/posix-functions/readdir.texi
+++ b/doc/posix-functions/readdir.texi
@@ -15,7 +15,8 @@ MSVC 14.
 On platforms where @code{off_t} is a 32-bit type, this function may not
 work correctly on huge directories larger than 2 GB.  Also, on platforms
 where @code{ino_t} is a 32-bit type, this function may report inode numbers
-incorrectly.  (Cf. @code{AC_SYS_LARGEFILE}.)
+incorrectly.  This can occur with file systems such as XFS (typically on
+large disks) and NFS.  (Cf. @code{AC_SYS_LARGEFILE}.)
 @end itemize
 
 Portability problems not fixed by Gnulib:
diff --git a/doc/posix-functions/readdir_r.texi 
b/doc/posix-functions/readdir_r.texi
index a56d390..6153448 100644
--- a/doc/posix-functions/readdir_r.texi
+++ b/doc/posix-functions/readdir_r.texi
@@ -25,5 +25,6 @@ Minix 3.1.8, mingw, MSVC 14.
 On platforms where @code{off_t} is a 32-bit type, this function may not
 work correctly on huge directories larger than 2 GB.  Also, on platforms
 where @code{ino_t} is a 32-bit type, this function may report inode numbers
-incorrectly.  The fix is to use the @code{AC_SYS_LARGEFILE} macro.
+incorrectly.  This can occur with file systems such as XFS (typically on
+large disks) and NFS.  The fix is to use the @code{AC_SYS_LARGEFILE} macro.
 @end itemize
diff --git a/doc/posix-functions/stat.texi b/doc/posix-functions/stat.texi
index 395717f..27fbe27 100644
--- a/doc/posix-functions/stat.texi
+++ b/doc/posix-functions/stat.texi
@@ -13,6 +13,11 @@ On platforms where @code{off_t} is a 32-bit type, 
@code{stat} may not correctly
 report the size of files or block devices larger than 2 GB.
 (Cf. @code{AC_SYS_LARGEFILE}.)
 @item
+On Linux/x86 and Linux/x86_64, applications compiled in 32-bit mode cannot
+access files that happen to have a 64-bit inode number.  This can occur with
+file systems such as XFS (typically on large disks) and NFS.
+(Cf. @code{AC_SYS_LARGEFILE}.)
+@item
 The @code{st_atime}, @code{st_ctime}, @code{st_mtime} fields are affected by
 the current time zone and by the DST flag of the current time zone on some
 platforms:
diff --git a/m4/largefile.m4 b/m4/largefile.m4
index 40b16bc..e381339 100644
--- a/m4/largefile.m4
+++ b/m4/largefile.m4
@@ -1,4 +1,5 @@
 # Enable large files on systems where this is not the default.
+# Enable support for files on Linux file systems with 64-bit inode numbers.
 
 # Copyright 1992-1996, 1998-2020 Free Software Foundation, Inc.
 # This file is free software; the Free Software Foundation
@@ -73,6 +74,9 @@ rm -rf conftest*[]dnl
 # one must use special compiler options to get large-file access to work.
 # For more details about this brain damage please see:
 # http://www.unix.org/version2/whatsnew/lfs20mar.html
+# Additionally, on Linux file systems with 64-bit inodes a file that happens
+# to have a 64-bit inode number cannot be accessed by 32-bit applications on
+# Linux x86/x86_64.  This can occur with file systems such as XFS and NFS.
 AC_DEFUN([AC_SYS_LARGEFILE],
 [AC_ARG_ENABLE(largefile,
                [  --disable-largefile     omit support for large files])