[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [bug-mailutils] Large file support in Mailutils

From: Alain Magloire
Subject: RE: [bug-mailutils] Large file support in Mailutils
Date: Thu, 22 Dec 2005 14:46:51 -0500

> -----Original Message-----
> From: address@hidden [mailto:bug-mailutils-
> address@hidden On Behalf Of Sergey Poznyakoff
> Sent: Thursday, December 22, 2005 7:11 AM
> To: address@hidden
> Subject: Re: [bug-mailutils] Large file support in Mailutils
> Hello,
> > First of all I'd like to announce that our custom mail.local based on
> > GNU Mailutils is operational on our production mail servers. Our mail
> > servers have about 200K users and we deliver ~1-1.5 million messages
> > to mailboxes daily.
> Great.


> > I compiled both my mda and GNU Mailutils (snapshot
> > 20051102) with -D_FILE_OFFSET_BITS=64. However this by itself was not
> > enough. I investigated the issue and the problem was the use of fseek(3)
> > instead of feeko(3), so I substituted all occurences of fseek() with
> > feeko() in Mailutils sources (found in
> > ./mailbox/file_stream.c,./lib/getpass.c,
> > ./mimeview/mimetypes-gram.c,./mimeview/mimetypes.y). This was enough
> > to fix the issues observed and have large file support.
> [..]
> > So I ask for these changes to be included upstream. Of course I am not
> > sure if this is enough to cover all possible issues involved.
> Of course we are aiming to supporting large files in mailutils. However,
> simply changing fseek to fseeko and ftell to ftello is not enough in
> general. There are more problems involved. Most probably these do not
> affect your installation, however they should be solved before releasing
> the package:

Glad to see, it worked for you,  this give us a good hint that we did not
rely too much on platform dependent behavior, thanks!
> 1. Using fseeko creates portability problems. It can, however, be solved
>    using appropriate autoconf magic.

Agreed, we need to pass the right flags to the compiler, is the OS support
this.  Question is not fseeko() and friends part of Posix now? Or whatever
XOpen standard?

> 2. The major problem is the use of off_t in the libmailutils. This data
>    type has different sizes depending on whether the library was
>    compiled with large file support or not. Thus, if one compiles
>    libmailutils with large file support and then links it to the program
>    that is compiled without that support, any mailutils functions taking
>    off_t as an argument will get incorrect data, which will lead, *in
>    the best case*, to a coredump. In the worst case it will trash whatever
>    user files involved, as well.
>    Compiling two variants of libmailutils (with and without large file
>    support) is an awkward solution. Therefore I have introduced
>    (2005-11-15) a new data type `mu_off_t' which is equivalent to 64-bit
>    off_t and have changed all related functions and datatypes
>    accordingly. This makes libmailutils safe to use no matter in what
>    combination it is linked, although this creates a small overhead in
>    the usual 32-bit mode.

It sounds good to me.  So basically if I read you right, you proposing to
always use 64 bits i.e. mu_off_t ==> 64 bits.

> 3. Most of the libmailutils code formats off_t for printing
>    using "%lu" format directive, which does not work for 64-bit data
>    types. Using "%llu" is impossible due to portability problems. The
>    recent snapshot (mailutils-20051216.tar.gz) solves this problem as
>    well. However, these changes are not committed yet, because I do not
>    quite like the way I implemented them. I am searching for a more
>    elegant solution.

Then the question is: where do we actually need to print/format mu_off_t?
On the top of my head, I do not remember any occurrences, so it may be a
non-issue.  And in the event that we need to print/format is it safe to
downcast it to an off_t? The other possibility is simply to always use our
auxiliary functions like asnprintf() that supports the format and to pass
the buffer down to printf() and all for printing.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]