[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [bug-mailutils] Large file support in Mailutils
From: |
Sergey Poznyakoff |
Subject: |
Re: [bug-mailutils] Large file support in Mailutils |
Date: |
Thu, 22 Dec 2005 12:11:26 EET |
Hello,
> First of all I'd like to announce that our custom mail.local based on
> GNU Mailutils is operational on our production mail servers. Our mail
> servers have about 200K users and we deliver ~1-1.5 million messages
> to mailboxes daily.
Great.
> I compiled both my mda and GNU Mailutils (snapshot
> 20051102) with -D_FILE_OFFSET_BITS=64. However this by itself was not
> enough. I investigated the issue and the problem was the use of fseek(3)
> instead of feeko(3), so I substituted all occurences of fseek() with
> feeko() in Mailutils sources (found in
> ./mailbox/file_stream.c,./lib/getpass.c,
> ./mimeview/mimetypes-gram.c,./mimeview/mimetypes.y). This was enough
> to fix the issues observed and have large file support.
[..]
> So I ask for these changes to be included upstream. Of course I am not
> sure if this is enough to cover all possible issues involved.
Of course we are aiming to supporting large files in mailutils. However,
simply changing fseek to fseeko and ftell to ftello is not enough in
general. There are more problems involved. Most probably these do not
affect your installation, however they should be solved before releasing
the package:
1. Using fseeko creates portability problems. It can, however, be solved
using appropriate autoconf magic.
2. The major problem is the use of off_t in the libmailutils. This data
type has different sizes depending on whether the library was
compiled with large file support or not. Thus, if one compiles
libmailutils with large file support and then links it to the program
that is compiled without that support, any mailutils functions taking
off_t as an argument will get incorrect data, which will lead, *in
the best case*, to a coredump. In the worst case it will trash whatever
user files involved, as well.
Compiling two variants of libmailutils (with and without large file
support) is an awkward solution. Therefore I have introduced
(2005-11-15) a new data type `mu_off_t' which is equivalent to 64-bit
off_t and have changed all related functions and datatypes
accordingly. This makes libmailutils safe to use no matter in what
combination it is linked, although this creates a small overhead in
the usual 32-bit mode.
3. Most of the libmailutils code formats off_t for printing
using "%lu" format directive, which does not work for 64-bit data
types. Using "%llu" is impossible due to portability problems. The
recent snapshot (mailutils-20051216.tar.gz) solves this problem as
well. However, these changes are not committed yet, because I do not
quite like the way I implemented them. I am searching for a more
elegant solution.
Regards,
Sergey