Re: [bug-gnu-libiconv] iconv tool buffers entire input file

[Yongzheng Wu]

On Fri, Dec 7, 2012 at 9:13 AM, Bruno Haible <address@hidden> wrote:
> Hi,
>
> Yongzheng Wu wrote:
>> The iconv tool buffers the entire input file in memory. This causes a
>> problem with big files. This problem has been raised by others before.
>> http://serverfault.com/questions/50919/how-to-run-command-on-parts-of-input-file
>> and http://keithdevens.com/weblog/archive/2011/Sep/22/iconv. I found a
>> comment in iconv_prog.c, "Since we have to deal with arbitrary
>> encodings we must read the whole text in a buffer and process it in
>> one step."
>
> Your report is about the 'iconv' program from GNU libc. Here we are on the
> mailing list for GNU libiconv, which is a different package. The 'iconv'
> program from GNU libiconv does not have the behaviour you report.
>
>> My question is, can this be fixed?
>
> Yes, this can be fixed, with fixed-size buffers and careful handling of
> the unprocessed bytes at the end of a block.
>
> I encourage you to report this to the GNU libc bug tracker, at
> http://sourceware.org/bugzilla/
>
> Bruno
>

Thank you for pointing it out. I just found out that the same bug was
filed 4 years ago there, and status is still "new".
http://sourceware.org/bugzilla/show_bug.cgi?id=6050

Yongzheng