[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Reducing iconv-induced memory usage

From: Ludovic Courtès
Subject: Reducing iconv-induced memory usage
Date: Tue, 26 Apr 2011 23:10:32 +0200
User-agent: Gnus/5.110015 (No Gnus v0.15) Emacs/23.3 (gnu/linux)


As Andy noted in the past, iconv conversion descriptors associated with
ports take up a lot of malloc’d memory, that only gets freed when
finalizers are run.  On GNU/Linux, a UTF-8 → UTF-8 C.D., which does
nothing, mallocs 180 KiB (!), according to the program attached.  So the
problem is acute.

So I think we should special-case UTF-8 I/O to not use iconv at all.

For output, it’s easy since we already do the conversion to UTF-8 in
‘display_string’.  For input, it’s a bit more work because input byte
streams have to be checked for invalid sequences.

I’m working on a patch but I’d like to get initial feedback and also
about whether it should wait until after 2.0.1 or not.


#include <iconv.h>
#include <malloc.h>

static size_t total;

static void * (*prev_hook) (size_t, const void *);

static void *
m (size_t s, const void *c)
  __malloc_hook = prev_hook;
  printf ("alloc %zi\n", s);
  void *r = malloc (s);
  total += s;
  __malloc_hook = &m;
  return r;

static void
my_init_hook (void)
  prev_hook = __malloc_hook;
  __malloc_hook = &m;

void (*__malloc_initialize_hook) (void) = my_init_hook;

main (int argc, char *argv[])
  total = 0;
  iconv_open ("UTF-8", "UTF-8");
  printf ("allocated %zi B\n", total);
  return 0;

reply via email to

[Prev in Thread] Current Thread [Next in Thread]