Re: [Chicken-users] A more detailed plea for help with FFI...

[Ed Watkeys]

On Dec 15, 2004, at 9:02 AM, felix winkelmann wrote:

> Well, sometimes handling memory allocation (or better resource-
> management) manually is not as bad as it sounds. The problems here
> are that finalizers can run at relatively unpredictable times: 
> especially
> when foreign data is finalized, one has to be absolutely sure that the
> resource isn't currently in use (in foreign land). Some go even so far
> and say that finalization without running it in a separate thread is
> inherently unsafe (There is some paper by Hans Boehm about this).
> Also, when wrapping libraries that do memory management manually,
> and which are designed around manual and controlled de-allocation,
> simple bolting a finalizer on the side might not always be the optimal
> solution.
> Personally, I try to avoid finalizers, unless there is really no other 
> way
> of doing things.

Yeah, this is not a simple, black-or-white issue. If, for example, you 
are writing code that opens files, your code should probably explicitly 
close them. (The enlightened lazy person in me just asked why. I'm not 
sure I have a good answer, if the GC can do it for me when the file 
becomes unreachable.)

Anyway, in this specific situation, the issue is not so much malloc- 
and free-ing resources but doing reference counting. Apple's Core 
Foundation and Cocoa code pervasively use a form of reference counting 
that solves this problem. If I pass a CGContextRef (a pointer to a 
graphics context) as an argument to a C (or Objective-C) function, that 
function will retain it using CFRetain (or by sending it a retain 
message). For example, in code to create a bitmap graphics context, you 
do something like this:

void *data = malloc(height * bytesPerRow);
assert(data);
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
CGImageAlphaInfo alphaInfo = kCGImageAlphaPremultipliedFirst;
CGContextRef cg_context = CGBitmapContextCreate(data, width, height,
                bitsPerComponent, bytesPerRow, colorspace, alphaInfo);
CFRelease(colorspace);

I allocated colorspace by calling CGColorSpaceCreateDeviceRGB(). It 
comes back with a retain count of one. I pass it to 
CGBitmapContextCreate(...), which increases it to two. I then tell 
colorspace to not stick around on my account using CFRelease(...) and 
its retain count gets reduced to one. I will then pass cg_context back 
to Chicken-land and set a finalizer that does the following:

void *data = CGBitmapContextGetData(cg_context);
CFRelease(cg_context);
if(data) free(data);

Yes, there's a theoretical possibility that someone else will hold a 
reference to the context when I release it and deallocate the data, 
which would lead to all sorts of badness, but in practice, it's not an 
issue. As far as I know, I won't be implementing anything that will 
increase a context's retain count. If I do wind up implementing 
something that does increase a context's retain count, then I guess 
it's time to start thinking about using a explicit, user-called 
finalizer.

This is the most complicated problem I'm confronting -- and it's not 
very complicated. The other cases are no-brainers. For example, I need 
to allocate memory for rectangle (CGRect) objects (simple structs of 
structs of floats), but they are always passed by value in the API. I 
can therefore safely blow them away with a finalizer. Requiring a user 
to explicitly destroy every rectangle would bring a mob with torches 
and pitchforks to my door.

Getting back to files for a moment, I don't see why we need explicit, 
user-called finalizers for files 99% percent of the time. Consider:

(define lines (read-lines (file-open "file-o-lines")))

Why shouldn't the GC system close the file for me? The alternative is 
writing something like this:

(define lines
  (let* ((file (file-open "file-o-lines))
         (lines (read-lines file)))
    (close-file file)
    lines))

Yuck! People are often going to write the first version, and it won't 
matter that much as long as it's not running in a loop. (OPEN_MAX is 
10240 on OS X.)

Now I guess one could write a macro that you could use like this:

(define lines (with-file read-lines "file-o-lines"))

Such as:

(define-macro (with-file thunk fname)
  `(let* ((file (file-open ,fname)) ; need a gensym here.
          (result (,thunk file)))   ; but not here.
    (close-file file)
    result))

Is it more appropriate to rely on GC or use macros to make this sort of 
thing convenient? There's a subject for heated discussion over drinks!

Is your "use finalizers conservatively" policy about performance or 
correctness? (That's not necessarily an either-or question.)

Ed