Re: lstat: Value too large...

[Darren Dunham]

> >> > cfengine:annorax: RecursiveCheck was looking at
> >> > /sse/annorax/knf/nervus/4663/pts0/nrva0000.avi when this
> >> >  happened:
> >> > cfengine:annorax: lstat: Value too large for defined data type

> > Sorta.  This appears to be a "large file" (bigger than 2G).  lstat() was
> > passed the filename, but lstat will fail on such a file.  'truss' could
> > be used to confirm that.  This limitation is there because regardless of
> > the OS and filesystems, old code would not be able to handle the return
> > values from such files.
> > 
> > If this is Solaris, I would assume that this binary was not compiled in
> > the large file compilation environment.  I don't know if cfengine
> > attempts to do that or not.

> I think it depends on whether the 64 bit OS is installed.

Not directly.  You do need OS support (Solaris 2.6 was the first with
"largefile" support), but it doesn't need to have a "64 bit OS"
installed or running. 

> The datastructures are not large enough in the 32bit OS
> to handle this.

They can be made to the correct size, but the code must be investigated
to make sure that it can cope with the (possibly larger) values that
might be returned.

> I don't think it matters whether it is stat
> or lstat, if that's what you mean. Either way, cfengine
> cannot do much about this...

There are 3 ways to usefully access large files in Solaris.

#1 If you are on a 64 bit OS, you can simply compile the program as a 64
   bit program.  It will probably work for most things, but it's
   limiting as this method restricts heavily where the binaries can run.
   32 bit programs will run in either OS, 64 bit only on 64 bit OSs.

#2 You can call 64 bit function calls explicitly.  Instead of lstat()
   there is lstat64().  This is the "transitional compilation
   environment".  It lets you make one or two functions largefile clean
   without looking at the rest of the code.

#3 You can compile in the "large file compilation environment".  Here
   all code references to functions like lstat() are translated into
   lstat64(), and if you use the correct header files, they are also
   correctly sized.  The way to use that environment is detailed in the
   lfcompile(5) man page.  I believe this is the general way to do large
   file compatibility on Solaris.

The benefit of #3 is that you don't have to make a lot of code changes.
However, the code does have to be reviewed to make sure that return
values are dealt with correctly.  You can't take the 'size' field from
the stat structure and assume that you can copy it into a 32 bit int and
not lose data.  That's where the code review is necessary unless it was
planned for large file compatibility from the start.

So as a first pass, I presume someone could attempt to build cfengine on
Solaris 2.6 or higher with _FILE_OFFSET_BITS=64 in the compile
environment and see if anything obvious breaks.

# cat foo.c
#include<sys/types.h>
#include<sys/stat.h>
main()
{
  char *file = "/tmp/file";
  struct stat statbuf;
  lstat(file,&statbuf);
}
# gcc foo.c
# truss ./a.out 2>&1 | grep lstat
lstat("/tmp/file", 0xFFBEFB60)                  Err#2 ENOENT
# gcc -D_FILE_OFFSET_BITS=64 foo.c
# truss ./a.out 2>&1 | grep lstat
lstat64("/tmp/file", 0xFFBEFB50)                Err#2 ENOENT
#

-- 
Darren Dunham                                           ddunham@taos.com
Unix System Administrator                    Taos - The SysAdmin Company
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >