[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [Chicken-users] RE: Win32/cygwin dynamic loading

From: Jonah Beckford
Subject: RE: [Chicken-users] RE: Win32/cygwin dynamic loading
Date: Sun, 23 Feb 2003 13:31:25 -0500

Oh yeah, just forget to make something explicit ... this dynamic loading
was just a proof of concept.  The goal was to find a set of principles
to support dynamic loading on the broadest of platforms (basically, code
normalization).  Now that I think we have most of the principles
(outlined below), I would still use __declspec(dllimport) for Win32
(which should work if the "principles" are followed) etc. and only use
the "primitive" shared loading technique for trully primitive platforms
and maybe for thread safety.


- A one-to-one mapping between units and shared libraries
- Runtime, as opposed to link-time, resolution of units (using
- Splitting of libchicken into a] libchicken (and all its variants like
libembedded, etc.) for the statically linked necessities of CHICKEN
(that is, all the CHICKEN_xxx functions and main/WinMain/DLLMain) and b]
libruntime which has the remainder of runtime.c.  Libruntime actually
has both a static version and a shared version, so conceptualize
libruntime as libruntime-static and libruntime-shared.
- The call to (load "xxx") for a dynamic module must not use any prefix
or suffix (like prefix=lib,  In fact, it will be difficult
to support any paths like (load "/usr/local/chicken/srfi-13") ... not
only is that system-dependent, the parsing routine used to search for
alternatives like /usr/local/chicken/cygsrfi-13-0.dll would be quite
difficult.  The dynamic modules must be locatable with a standard
mechanism (maybe search the current directory, followed by the
system-dependant library search path, followed by CHICKEN_HOME).  This
also means they must be installable with a standard mechanism (I like
William Annis' suggestion of something like Python's Distutils or Perl's
Makefile.PL; I haven't looked at "eggs" yet, but I have a suspicion they
are for pure Scheme code).
- Placing a FUNC() wrapper around the functions in runtime.c, so that
function names can be mangled if on a primitive platform
- Specification of -static or -shared when using chicken-config (so it
can pick the right libruntime) and change the compiler -D defines.  This
also means the chicken-config, or its replacement, must be useable on
all platforms.
- Using temporary variables and constants to iterate over arrays.
    /* Mark collectibles: */
    if((msp = C_collectibles) != NULL)
      while(*msp != NULL) mark(*(msp++));
Needs 2 modifications:
  for(i = 0; i < C_symbol_table_size; C_symbol_table[ i++ ] =
Corrected version of above:
  C_word *sym; /* access CHICKEN arrays with a temporary pointer */
  const int TABLE_SIZE = C_symbol_table_size; /* don't check
C_symbol_table_size every iter */
  for(i = 0, sym = C_symbol_table; i < TABLE_SIZE; *(sym++) =


-----Original Message-----
From: Jonah Beckford [mailto:address@hidden 
Sent: February 21, 2003 10:56 PM
To: 'Jonah Beckford'; 'felix'
Cc: 'Felix Winkelmann'; 'address@hidden'
Subject: RE: [Chicken-users] RE: Win32/cygwin dynamic loading

Changed web site to a permanent location ... is now:

(and SWIG CHICKEN is at

MinGW has also been tested with the dynamic loading (use the script).


-----Original Message-----
From: Jonah Beckford [mailto:address@hidden 
Sent: February 21, 2003 1:59 AM
To: 'felix'
Cc: 'Felix Winkelmann'; 'address@hidden'
Subject: RE: [Chicken-users] RE: Win32/cygwin dynamic loading

I have the modifications to Chicken that allow dynamic loading of
modules in a portable fashion.  Right now only cygwin has been tested
(mingw had also been, but not lately).  If you have cygwin, then run the script first because the bootstrapping process is not easy
to do by hand.

The web site is up most but not all of the time.

Major changes:
- (declare (uses xx yy)) is basically translated into (load "xx-unit")
(load "yy-unit") conditionally at the C source code level.
- while not enforced, the idea is that 1 unit == 1 dynamic module.  No
bundling of many units into one shared library (although you still may
if you don't care about portability).
- (load "xx-unit") will look for xx-unit, xx-unit.dll, cygxx-unit.dll
and libxx-unit.dll on cygwin.  It does .so and .sl and its variants as
well, just hasn't been tested on those platforms yet.  If can't find a
module, then will proceed with the original Chicken rules (like looking
for a xx-unit.scm).
- on "primitive shared" platforms, all function calls and variable
accesses that are defined in chicken.h are done through a data structure
(of function pointers and variables).  This is done transparently
through macros.  So a function call C_string(x,y,z) is macro expanded to
(*(C_shlib->priv_C_string))(x,y,z) and a variable access to
C_collectibles is macro expanded to (*(C_shlib->priv_C_collectibles)).
When a unit is loaded through (load "xx-unit"), its own C_shlib is
initialized with the proper values.  Basically, we simulate the
indirection method used by ordinary DLLs, but we define it in a way
(through C_shlib) such that it works on most platforms.
- there is a script that writes all the macros and all the
code for you ... just add a new variable or function to chicken.h, run and nothing else to do.
- MinGW is done through the ./configure script, not makefile.mingw.

- Should work on all shared library platforms because each xx-unit is
completely self-contained.  In fact, the only requirement is that the
platform have some object file format that supports relocatable code
(ie. gcc -PIC) and a predefined entry point (either a symbol or an
absolute address).
- Can have circular dependencies between xx-unit and yy-unit because the
units are dynamically loaded.
- Quite unexpectedly, the technique can be used to make Chicken thread
safe as well (a thread can get its own C_shlib).  The remaining static
variables in runtime.c would have to be moved to chicken.h (trivial;
I've already done half), the static C_[xxx_]toplevel_initialized in
Chicken generated code would have to be removed (trivial) and each
C_shlib access would have to go into thread local storage (not trivial).

- Is slower because of the pointer dereference of C_shlib.  This becomes
an issue inside loops (good example: looping over C_collectibles).  The
C code developer has to be careful and introduce temporary variables
above large loops.  With the temporaries (which is nothing more than a
hint to the C compiler to not dereference the pointer every step in a
loop), the C_shlib method would be the exact same speed as access
__declspec(dllimport/export) variables.
- I'm sure there are more ...


-----Original Message-----
From: felix [mailto:address@hidden 
Sent: February 18, 2003 4:02 PM
To: Jonah Beckford
Cc: 'Felix Winkelmann'; address@hidden
Subject: Re: [Chicken-users] RE: Win32/cygwin dynamic loading

Jonah Beckford wrote:
> I took your comment about solution #2 into mind ... I don't need to 
> pass in the toplevel pointer.  I did need to split up libchicken into 
> two libraries though (a static library that has defns that must be 
> statically linked [CHICKEN_initialize/run/invoke, main/WinMain/DLLMain

> and global data], and a runtime library for everything else).

Sounds reasonable. But we still have the option of generating `main'
entry-points into compiled code. Yet, the method youu describe seems

> Also, my 'better' solution to #3 is basically to resolve the
> C_<unit>_toplevel at runtime instead of link time.  Circular 
> dependencies should be fine using the new method; no need to touch the

> srfi-37 unit :)

Too late. I removed the serialization stuff from the lolevel unit and
srfi-37 does the option-parsing by hand (which is more efficient

I'm checking in the changes just at this moment...

> Also, I was using a Win32 dload_2 stub that called
> LoadLibrary/GetProcAddress.  This works on all Win32 platforms ... 
> Cygwin does have a dlopen/dlsym, but it is just a tiny wrapper around 
> LoadLibrary.


> It's mostly finished ... I will tar.gz it up and make it available for
> testing in a day or two.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]