Re: [Chicken-users] Best way to share memory between C and Chicken

[Ivan Raikov]

I cannot answer your question about performance, but you have prompted me to create a simple example with POSIX shared memory. Attached are two files, producer.c and consumer.scm. The producer (a C program) writes strings to a POSIX shared memory file, and the consumer (a Scheme program) reads them, prints them and then unlinks the buffer. The producer uses mmap, as Dan mentioned in his reply. This may be clunky, but there is no need to worry about synchronization.

Using e.g. Linux shmget for shared memory between threads _might_  give you higher performance, but then you have to mess around with semaphores and so on. I will not have time to adapt the code you link to, but I can help  if you are having trouble with a particular construct in posix-shm.

   -Ivan

 

 

 

On Sun, May 5, 2013 at 8:07 AM, Jörg F. Wittenberger <address@hidden> wrote:

On May 4 2013, Ivan Raikov wrote:

 I think you can try to have native threads by running a separate instance
of the Chicken runtime for each thread, but are you sure that you will
really achieve a significant speedup over Unix processes and/or MPI?
It's not the early nineties anymore... I suggest prototyping  code and
running real-world benchmarks first.

Hm.  I dunno, I'm old school.

So far I have at least one case, where I've been resorting to native
threads because I didn't see another solution.  Maybe you/anybody
could enlighten me?

The case at hand: I'm running sqlite queries, which can take a long time. I don't want to block the chicken system while those are busy. Worse: I'm using the sqlite VFS to get the actual data blocks ... from chicken, hence the chicken thread MUST not wait for sqlite... (for reference an old version of the code is here http://lists.nongnu.org/archive/html/chicken-users/2010-01/msg00046.html )

As I said: I'm old school - could I really expect a reasonable performance
from running a separate sqlite process and cop the data back and forth
between the processes?  Suggestion how to do this are welcome!

On Sat, May 4, 2013 at 8:39 AM, Dan Leslie <address@hidden> wrote:

...

Relatedly, is anyone poking at implementing native threads?
I've been digging around a bit but haven't had much time to progress very
far.

In fact I'm contemplating about this for a while and never got the
courage to try.  (Especially because I don't see how I could create
a half-way meaningful benchmark beforehand to judge the gain/loss
early in the development process.)

The bottleneck for me is usually blocking on i/o.  Wrt. this I do not
expect (as in wild guess - no benchmarks done) much from multiple
threads.

However some things I'm doing are just computations.  Am I thinking
"early nineties" style when I assume that chicken will utilize only
one cpu core?  If so, I'd prefer not spare the others all the time.

My problem: I can't see a good way to share chicken-managed memory
between threads.  I recall having seen a paper on using a dedicated
thread for memory allocation...but can't find it anymore.

Nevertheless: maybe it would be feasible to do all second gen
gc in a dedicated thread and force a minor gc before passing data
between threads?  Would it?


Best

/Jörg



........

producer.c
Description: Text Data

consumer.scm
Description: Binary data