file size limit

[Daniel Heiserer]

referring to
http://www.octave.org/mailing-lists/octave-maintainers/2003/380

However if we implement a int64_t as the file positioning and take care 
of the mapping betweeWith gcc 3.4, streamoff and streampos are classes 
that hold 64-bit
ints (either long long or int64_t).  But they are not simple typedefs
for either of these types, so it is still not possible to write
something like

  streampos pos = ...;

  long long ftell_retval = (long long) pos;

because there is no way to cast a streampos or streamoff type to an
integer.

For portability, we can't assume that these types are any particular
integer value.  Different systems are free to implement streamoff and
streampos (or fpos_t if you are using C) in any way they choose
(provided that they meet the rather limited constraints of the
respective standards).

Note that the MathWorks is going to face similar problems if they wish
to support large files in Matlab.
-----------------------------------------------------------------
my comments:
If these are classes they should have an interface to set and get the value.
If an int64_t is used as the positioning it should last for another 15 
years.

http://www.cplusplus.com/ref/iostream/streampos.html states:
streampos       
  cplusplus.com
(stream position type)
Type to express absolute positions within streams.
  This type describes a class to contain all the information needed to 
restore an arbitrary file-position indicator within a stream. It can be 
constructed from or casted to an integer offset value (streamoff).
See also.
  iostream library
For me that means that if octave uses a
std::vector<int> it will always be capable of positioning a file.
All is needed is a forward and backward translation of the file position.

-- daniel

-----------------------------------------------------------------
> | I try to load a file with 8GB into octave.
> | 
> | Unfortunately it immediately crashes once it hits the 2GB barrier:
> | -----------------------------------------------------
> | octave:1> load ftn51.mat
> | panic: Segmentation fault -- stopping myself...
> | attempting to save variables to `octave-core'...
> | save to `octave-core' complete
> | Segmentation fault
> | -----------------------------------------------------
> | My machine has 24GB main memory.
> | The file has 8GB:
> | 8153243903 Sep 27 14:22 ftn51.mat
> | 
> | 
> | I used octave 2.1.44 on Itanium2, Linux 2.4.22-11.msc-smp ia64
> | However I will try 2.1.57 now.
>
> It won't help.  This is a known problem, with no quick fix.  However,
> there have been some discussions about how to start fixing it (see the
> maintainers list).  So far, no one has started to do the work in a way
> that will be likely to be included in a new snapshot or release.