Re: [Tinycc-devel] TCC: Best way to release functions that has been compiled in memory?

[Basile Starynkevitch]

On Wed, 2014-01-01 at 20:25 +0800, 武振伟 wrote:
> Hi!
> I'm working on making a script interpreter, and find TinyCC is just
> for me. Compiling string into a function pointer is really cool! But I
> wonder if there's a way to release those functions? Because there may
> be a lot of functions, and some may change and must be recompiled for
> many times. I think I can't just keep all these in memory. What should
> I do? I know I can compile them into DLL, but it seems not as
> convenient as compiling in memory. Are there other ways to solve this?


In practice, on current Linux desktop machines at least, it does not
matter much. Indeed, you are leaking a little code memory. But in
practice, that does not matter much (because unused code becomes cold
pages and eventually gets swapped out, without eating RAM anymore, only
swap and file space).

I have on my home page a self-contained program
http://starynkevitch.net/Basile/manydl.c which illustrates the point. It
is generating a pseudorandom C code, then compiling it into a shared
object and dlopen-ing it. I was able to run it several days without any
issue, and it could dlopen many hundred thousands of shared objects
without apparent pain (It does not call dlclose before the end). In
practice a dlopen-ed shared object (for some generated C code) has often
a few dozen kilobytes of code, you don't care that much loosing a few
megabytes of code (which amounts to thousands of dlopen).

Likewise my MELT [MELT is a Lispy domain specific language to extend or
customize GCC] program http://gcc-melt.org/ is generating C++ code (for
GCC internals) and compiling it and dlopen-ing it without ever
dlclose-ing it. In practice it does not matter much.

When you generate C code, most of the time is spent in compiling it (ok,
TinyCC compiles about 10 times faster than GCC, but the produced machine
code is often 2 or 3 times slower than what gcc -O1 -fPIC is producing;
so it could be worth to sometimes use GCC and not always TinyCC, YMMV).
So the limiting factor is the time for generating the C code and the
time for compiling it. So in practice, for a process running less than a
dozen hours of CPU time, you won't generate more than a few dozen
thousands of C files (i.e. dlopen-ing the same amount of shared objects)
and not freeing (dlclose-ing) the code is not a real issue (on a desktop
running Linux). Of course, if you consider server programs, it becomes
an issue (unless you restart the server every day).

Notice that the main issue is not about how to release code (with
dlopen-ed shared object, it is just dlclose!). The main issue is to know
which code you can safely release, i.e. which code has no active call
stacks on the call frame and would never be called anymore (i.e. no
closures of your language point to that code). It is a matter of garbage
collecting code, and that could be quite hard for generic C code.

If you dlclose (or release) some code which has some active call frame
below the current one, you would crash (with SIGSEGV) when returning
into that call frame.

Regards, and happy new 2014 year!
-- 
Basile STARYNKEVITCH         http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net mobile: +33 6 8501 2359
8, rue de la Faiencerie, 92340 Bourg La Reine, France
*** opinions {are only mine, sont seulement les miennes} ***