[Gcl-devel] Re: OpenBSD and GCL

[Magnus Henoch]

Camm Maguire <address@hidden> writes:

> Greetings!
>
> 1) I've put in what I hope are analogs to your very helpful patches
>    regarding data seg size limits and mallocing error messages.
>    Please check them out and let me know if I've overlooked anything.
>    I'm assuming without any shred of evidence that fputs won't malloc
>    from other calls in the existing code.

It doesn't work as commited; same symptoms as before.  I tested the
following program on Debian and OpenBSD:

#include <stdio.h>

#define USE_FPUTS 0

void *malloc(void)
{
#if USE_FPUTS
  fputs("recursion\n", stderr);
#else
  write(1, "in malloc\n", 10);
#endif
  return NULL;
}

int main(void)
{
  fputs("start\n", stderr);
  return 0;
}

On Debian, only "start" is printed, but on OpenBSD, both "in malloc"
and "start" are printed.  gdb confirms that something calls malloc
before main is reachead - that's why I put the setrlimit call in
alloc.c instead of main.c.  Anyway, in both cases it seems that fputs
does not use malloc.

Then, setting USE_FPUTS to 1, I get the same result as with GCL -
malloc uses fputs, which reaches into some library without debugging
symbols, which calls malloc, which calls fputs... etc.

After digging around a bit in OpenBSD libc, it seems to me that stdio
will happily use unbuffered I/O if it can't allocate memory, but it
needs the return value to notice that.  Like this:

1. Something (still not sure what) makes stdio try to allocate a
   buffer.
2a. My fake malloc returns NULL - stdio uses unbuffered I/O and is
   happy with that.
2b. My fake malloc calls fputs with stderr, which has no buffer yet -
   goto 1.

Thus, to avoid recursion the malloc of GCL must never use stdio to
report failure.  (OpenBSD specific, that is, as stdio of glibc seems
not to use malloc)

And, if GCL malloc is to fail ungraciously without giving the option
of retrying later (which seems to me quite sane), the setrlimit call
needs to happen as soon as possible - in OpenBSD's case before main.

> 2) Re EFAULT in writing saved_pre_gcl --  this is almost certainly in
>    unexec.  The code is taken straight from emacs, so if OpenBSD has a
>    working emacs, we should have a working solution here too.

Thanks for the pointer, I'll see what I find.

Regards,
Magnus