[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Guile-commits] 01/01: Replace libltdl with raw dlopen, dlsym
|
From: |
Andy Wingo |
|
Subject: |
[Guile-commits] 01/01: Replace libltdl with raw dlopen, dlsym |
|
Date: |
Fri, 29 Jan 2021 10:14:47 -0500 (EST) |
wingo pushed a commit to branch excise-ltdl
in repository guile.
commit 9e6ac923bfdcc89b1b97851f9be53371bf4b5b86
Author: Andy Wingo <wingo@pobox.com>
AuthorDate: Fri Jan 22 16:39:11 2021 +0100
Replace libltdl with raw dlopen, dlsym
* NEWS: Update.
* am/bootstrap.am (SOURCES):
* module/Makefile.am (SOURCES): Add system/foreign-library.scm.
* configure.ac: Replace ltdl check with -ldl check.
* libguile/dynl.c: Rewrite to just expose core dlopen / dlsym / etc to a
helper Scheme module.
(scm_dynamic_link, scm_dynamic_pointer, scm_dynamic_function)
(scm_dynamic_object_p, scm_dynamic_call): Rewrite in terms of (system
foreign-library).
* libguile/extensions.c (load_extension): Avoid scm_dynamic_call.
* module/system/foreign-library.scm: New file.
* module/oop/goops.scm (<dynamic-object>): Hackily export
<foreign-library> instead of a class here.
* doc/ref/api-foreign.texi (Foreign Function Interface): Rewrite to only
document the new interfaces. Eventually we will deprecate
dynamic-link and friends.
* doc/ref/guile.texi (API Reference): Move Foreign Objects after Foreign
Function Interface. Seems there should be some closer relationship
but this will do for now.
* doc/ref/tour.texi (Putting Extensions into Modules):
* doc/ref/libguile-parallel.texi (Parallel Installations): Update for
rename of Modules and Extensions to Foreign Extensions.
* libguile/deprecated.h:
* libguile/deprecated.c (scm_dynamic_unlink): Deprecate.
* libguile/guile.c: Remove ltdl include.
* test-suite/tests/foreign.test: Update tests to use new API, and update
error expectations.
---
NEWS | 55 +++
am/bootstrap.am | 3 +-
configure.ac | 8 +-
doc/ref/api-foreign.texi | 956 +++++++++++++++++---------------------
doc/ref/guile.texi | 10 +-
doc/ref/libguile-parallel.texi | 6 +-
doc/ref/tour.texi | 4 +-
libguile/deprecated.c | 17 +-
libguile/deprecated.h | 4 +-
libguile/dynl.c | 427 +++++------------
libguile/dynl.h | 9 +-
libguile/extensions.c | 7 +-
libguile/guile.c | 3 +-
module/Makefile.am | 4 +-
module/oop/goops.scm | 9 +-
module/system/foreign-library.scm | 231 +++++++++
test-suite/tests/foreign.test | 36 +-
17 files changed, 906 insertions(+), 883 deletions(-)
diff --git a/NEWS b/NEWS
index 234885d..2f55f5d 100644
--- a/NEWS
+++ b/NEWS
@@ -7,6 +7,46 @@ Please send Guile bug reports to bug-guile@gnu.org.
Changes in 3.0.6 (since 3.0.5)
+* Notable changes
+
+** Reimplement dynamic library loading ("dlopening") without libltdl
+
+Guile used to load dynamic libraries with libltdl, a library provided by
+the Libtool project.
+
+Libltdl provided some compatibility benefits when loading shared
+libraries made with older toolchains on older operating systems.
+However, no system from the last 10 years or so appears to need such a
+thick compatibility layer.
+
+Besides being an unmaintained dependency of limited utility, libltdl
+also has the negative aspect that in its search for libraries to load,
+it could swallow useful errors for libraries that are found but not
+loadable, instead showing just errors for search path candidates that
+are not found.
+
+Guile now implements dynamic library loading directly in terms of the
+standard "dlopen" interface, providing a limited shim for platforms with
+similar functionality exposed under different names (MinGW).
+
+This change has a few practical impacts to Guile users. There is a new
+library search path variable, `GUILE_EXTENSIONS_DIR'. Also, errors when
+loading a library fails now have better errors. And Guile no longer has
+a libltdl dependency.
+
+Although Guile no longer uses libltdl, for backwards compatibility Guile
+still adds `LTDL_LIBRARY_PATH' to the loadable library search path, and
+includes ad-hoc logic to support uninstalled dynamically loadable
+libraries via also adding the ".libs" subdirectories of
+`LTDL_LIBRARY_PATH' elements. See "Foreign Libraries" in the
+documentation for a full discussion.
+
+** Updated Gnulib
+
+The Gnulib compatibility library has been updated, for the first time
+since 2017 or so. We expect no functional change but look forward to
+any bug reports.
+
* New interfaces and functionality
** `call-with-port'
@@ -17,6 +57,21 @@ See "Ports" in the manual.
See "Bytevector Ports" in the manual.
+** `GUILE_EXTENSIONS_DIR' environment variable.
+
+** `(system foreign-library)' module
+
+See the newly reorganized "Foreign Function Interface", for details.
+These new interfaces replace `dynamic-link', `dynamic-pointer' and
+similar, which will eventually be deprecated.
+
+* New deprecations
+
+** `dynamic-unlink'
+
+This function now has no effect; Guile will not unload dynamically
+linked modules, as that can destabilize the system.
+
* Incompatible changes
** `call-with-output-string' closes port on normal exit
diff --git a/am/bootstrap.am b/am/bootstrap.am
index 2821304..acc00c7 100644
--- a/am/bootstrap.am
+++ b/am/bootstrap.am
@@ -1,4 +1,4 @@
-## Copyright (C) 2009-2020 Free Software Foundation, Inc.
+## Copyright (C) 2009-2021 Free Software Foundation, Inc.
##
## This file is part of GNU Guile.
##
@@ -120,6 +120,7 @@ SOURCES = \
system/vm/program.scm \
system/vm/vm.scm \
system/foreign.scm \
+ system/foreign-library.scm \
\
language/tree-il/compile-cps.scm \
language/tree-il/cps-primitives.scm \
diff --git a/configure.ac b/configure.ac
index bc7bfc6..8166a56 100644
--- a/configure.ac
+++ b/configure.ac
@@ -105,12 +105,8 @@ AC_PROG_LIBTOOL
AM_CONDITIONAL([HAVE_SHARED_LIBRARIES], [test "x$enable_shared" = "xyes"])
-dnl Check for libltdl.
-AC_LIB_HAVE_LINKFLAGS([ltdl], [], [#include <ltdl.h>],
- [lt_dlopenext ("foo");])
-if test "x$HAVE_LIBLTDL" != "xyes"; then
- AC_MSG_ERROR([GNU libltdl (Libtool) not found, see README.])
-fi
+# Some systems provide dlopen via libc; others require -ldl.
+AC_SEARCH_LIBS([dlopen], [dl])
AC_CHECK_PROG(have_makeinfo, makeinfo, yes, no)
AM_CONDITIONAL(HAVE_MAKEINFO, test "$have_makeinfo" = yes)
diff --git a/doc/ref/api-foreign.texi b/doc/ref/api-foreign.texi
index b0d6c24..e4c0684 100644
--- a/doc/ref/api-foreign.texi
+++ b/doc/ref/api-foreign.texi
@@ -1,6 +1,6 @@
@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
-@c Copyright (C) 1996, 1997, 2000-2004, 2007-2014, 2016-2017
+@c Copyright (C) 1996, 1997, 2000-2004, 2007-2014, 2016-2017, 2021
@c Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@@ -9,245 +9,208 @@
@cindex foreign function interface
@cindex ffi
-The more one hacks in Scheme, the more one realizes that there are
-actually two computational worlds: one which is warm and alive, that
-land of parentheses, and one cold and dead, the land of C and its ilk.
-
-But yet we as programmers live in both worlds, and Guile itself is half
-implemented in C. So it is that Guile's living half pays respect to its
-dead counterpart, via a spectrum of interfaces to C ranging from dynamic
-loading of Scheme primitives to dynamic binding of stock C library
-procedures.
+Sometimes you need to use libraries written in C or Rust or some other
+non-Scheme language. More rarely, you might need to write some C to
+extend Guile. This section describes how to load these ``foreign
+libraries'', look up data and functions inside them, and so on.
@menu
-* Foreign Libraries:: Dynamically linking to libraries.
-* Foreign Functions:: Simple calls to C procedures.
-* C Extensions:: Extending Guile in C with loadable modules.
-* Modules and Extensions:: Loading C extensions into modules.
-* Foreign Pointers:: Accessing global variables.
-* Dynamic FFI:: Calling arbitrary C functions.
+* Foreign Libraries:: Dynamically linking to libraries.
+* Foreign Extensions:: Extending Guile in C with loadable modules.
+* Foreign Pointers:: Pointers to C data or functions.
+* Foreign Types:: Expressing C types in Scheme.
+* Foreign Functions:: Simple calls to C procedures.
+* Void Pointers and Byte Access:: Pointers into the ether.
+* Foreign Structs:: Packing and unpacking structs.
+* More Foreign Functions:: Advanced examples.
@end menu
@node Foreign Libraries
@subsection Foreign Libraries
-Most modern Unices have something called @dfn{shared libraries}. This
-ordinarily means that they have the capability to share the executable
-image of a library between several running programs to save memory and
-disk space. But generally, shared libraries give a lot of additional
-flexibility compared to the traditional static libraries. In fact,
-calling them `dynamic' libraries is as correct as calling them `shared'.
-
-Shared libraries really give you a lot of flexibility in addition to the
-memory and disk space savings. When you link a program against a shared
-library, that library is not closely incorporated into the final
-executable. Instead, the executable of your program only contains
-enough information to find the needed shared libraries when the program
-is actually run. Only then, when the program is starting, is the final
-step of the linking process performed. This means that you need not
-recompile all programs when you install a new, only slightly modified
-version of a shared library. The programs will pick up the changes
-automatically the next time they are run.
-
-Now, when all the necessary machinery is there to perform part of the
-linking at run-time, why not take the next step and allow the programmer
-to explicitly take advantage of it from within their program? Of course,
-many operating systems that support shared libraries do just that, and
-chances are that Guile will allow you to access this feature from within
-your Scheme programs. As you might have guessed already, this feature
-is called @dfn{dynamic linking}.@footnote{Some people also refer to the
-final linking stage at program startup as `dynamic linking', so if you
-want to make yourself perfectly clear, it is probably best to use the
-more technical term @dfn{dlopening}, as suggested by Gordon Matzigkeit
-in his libtool documentation.}
-
-We titled this section ``foreign libraries'' because although the name
-``foreign'' doesn't leak into the API, the world of C really is foreign
-to Scheme -- and that estrangement extends to components of foreign
-libraries as well, as we see in future sections.
-
-@deffn {Scheme Procedure} dynamic-link [library]
-@deffnx {C Function} scm_dynamic_link (library)
-Find the shared library denoted by @var{library} (a string) and link it
-into the running Guile application. When everything works out, return a
-Scheme object suitable for representing the linked object file.
-Otherwise an error is thrown. How object files are searched is system
-dependent.
-
-Guile first tries to load @var{library} as the absolute file name of a shared
-library. If that fails, it then falls back to interpret
-@var{library} as just the name of some shared library that will be
-searched for in the places where shared libraries usually reside, such
-as @file{/usr/lib} and @file{/usr/local/lib}.
-
-@var{library} should not contain an extension such as @code{.so}, unless
-@var{library} represents the absolute file name to the shared library. The
-correct file name extension for the host operating system is provided
-automatically, according to libltdl's rules (@pxref{Libltdl interface,
-lt_dlopenext, @code{lt_dlopenext}, libtool, Shared Library Support for
-GNU}).
-
-When @var{library} is omitted, a @dfn{global symbol handle} is returned. This
-handle provides access to the symbols available to the program at run-time,
-including those exported by the program itself and the shared libraries already
-loaded.
-
-Note that on hosts that use dynamic-link libraries (DLLs), the global
-symbol handle may not be able to provide access to symbols from
-recursively-loaded DLLs. Only exported symbols from those DLLs directly
-loaded by the program may be available.
-@end deffn
+Just as Guile can load up Scheme libraries at run-time, Guile can also
+load some system libraries written in C or other low-level languages.
+We refer to these as dynamically-loadable modules as @dfn{foreign
+libraries}, to distinguish them from native libraries written in Scheme
+or other languages implemented by Guile.
+@cindex foreign libraries
+@cindex libraries, foreign
+
+Foreign libraries usually come in two forms. Some foreign libraries are
+part of the operating system, such as the compression library
+@code{libz}. These shared libraries are built in such a way that many
+programs can use their functionality without duplicating their code.
+When a program written in C is built, it can declare that it uses a
+specific set of shared libraries.
+@cindex shared libraries
+@cindex libraries, shared
+When the program is run, the operating system takes care of locating and
+loading the shared libraries.
+
+The operating system components that can dynamically load and link
+shared libraries when a program is run are also available
+programmatically during a program's execution. This is the interface
+that's most useful for Guile, and this is what we mean in Guile when we
+refer to @dfn{dynamic linking}. Dynamic linking at run-time is
+sometimes called @dfn{dlopening}, to distinguish it from the dynamic
+linking that happens at program start-up.
+@cindex dynamic linking
+@cindex dlopening
+
+The other kind of foreign library is sometimes known as a module,
+plug-in, bundle, or an extension. These foreign libraries aren't meant
+to be linked to by C programs, but rather only to be dynamically loaded
+at run-time -- they extend some main program with functionality, but
+don't stand on their own. Sometimes a Guile library will implement some
+of its functionality in a loadable module.
+
+In either case, the interface on the Guile side is the same. You load
+the interface using @code{load-foreign-library}. The resulting foreign
+library object implements a simple lookup interface whereby the user can
+get addresses of data or code exported by the library. There is no
+facility to inspect foreign libraries; you have to know what's in there
+already before you look.
+
+Routines for loading foreign libraries and accessing their contents are
+implemented in the @code{(system foreign-library)} module.
-@deffn {Scheme Procedure} dynamic-object? obj
-@deffnx {C Function} scm_dynamic_object_p (obj)
-Return @code{#t} if @var{obj} is a dynamic library handle, or @code{#f}
-otherwise.
-@end deffn
-
-@deffn {Scheme Procedure} dynamic-unlink dobj
-@deffnx {C Function} scm_dynamic_unlink (dobj)
-Unlink the indicated object file from the application. The
-argument @var{dobj} must have been obtained by a call to
-@code{dynamic-link}. After @code{dynamic-unlink} has been
-called on @var{dobj}, its content is no longer accessible.
-@end deffn
-
-@smallexample
-(define libgl-obj (dynamic-link "libGL"))
-libgl-obj
-@result{} #<dynamic-object "libGL">
-(dynamic-unlink libGL-obj)
-libGL-obj
-@result{} #<dynamic-object "libGL" (unlinked)>
-@end smallexample
-
-As you can see, after calling @code{dynamic-unlink} on a dynamically
-linked library, it is marked as @samp{(unlinked)} and you are no longer
-able to use it with @code{dynamic-call}, etc. Whether the library is
-really removed from you program is system-dependent and will generally
-not happen when some other parts of your program still use it.
-
-When dynamic linking is disabled or not supported on your system,
-the above functions throw errors, but they are still available.
-
-
-@node Foreign Functions
-@subsection Foreign Functions
+@example
+(use-modules (system foreign-library))
+@end example
-The most natural thing to do with a dynamic library is to grovel around
-in it for a function pointer: a @dfn{foreign function}.
-@code{dynamic-func} exists for that purpose.
-
-@deffn {Scheme Procedure} dynamic-func name dobj
-@deffnx {C Function} scm_dynamic_func (name, dobj)
-Return a ``handle'' for the func @var{name} in the shared object referred to
-by @var{dobj}. The handle can be passed to @code{dynamic-call} to
-actually call the function.
-
-Regardless whether your C compiler prepends an underscore @samp{_} to the
global
-names in a program, you should @strong{not} include this underscore in
-@var{name} since it will be added automatically when necessary.
+@deffn {Scheme Procedure} load-foreign-library [library] @
+ [#:extensions=system-library-extensions] @
+ [#:search-ltdl-library-path?=#t] @
+ [#:search-path=search-path] @
+ [#:search-system-paths?=#t] [#:lazy?=#t] [#:global=#f]
+Find the shared library denoted by @var{library} (a string or @code{#f})
+and link it into the running Guile application. When everything works
+out, return a Scheme object suitable for representing the linked object
+file. Otherwise an error is thrown.
+
+If @var{library} argument is omitted, it defaults to @code{#f}. If
+@code{library} is false, the resulting foreign library gives access to
+all symbols available for dynamic linking in the main binary.
+
+It is not necessary to include any extension such as @code{.so} in
+@var{library}. For each system, Guile has a default set of extensions
+that it will try. On GNU systems, the default extension set is just
+@code{.so}; on Windows, just @code{.dll}; and on Darwin (Mac OS), it is
+@code{.bundle}, @code{.so}, and @code{.dylib}. Pass @code{#:extensions
+@var{extensions}} to override the default extensions list. If
+@var{library} contains one of the extensions, no extensions are tried,
+so it is possible to specify the extension if you know exactly what file
+to load.
+
+Unless @var{library} denotes an absolute file name or otherwise contains
+a directory separator (@code{/}, and also @code{\} on Windows), Guile
+will search for the library in the directories listed in
+@var{search-paths}. The default search path has three components, which
+can all be overriden by colon-delimited (semicolon on Windows)
+environment variables:
+
+@table @env
+@item GUILE_EXTENSIONS_PATH
+This is the main environment variable for users to add directories
+containing Guile extensions. The default value has no entries. This
+environment variable was added in Guile 3.0.6.
+@item LTDL_LIBRARY_PATH
+Before Guile 3.0.6, Guile loaded foreign libraries using @code{libltdl},
+the dynamic library loader provided by libtool. This loader used
+@env{LTDL_LIBRARY_PATH}, and for backwards compatibility we still
+support that path.
+
+However, @code{libltdl} would not only open @code{.so} (or @code{.dll}
+and so on) files, but also the @code{.la} files created by libtool. In
+installed libraries -- libraries that are in the target directories of
+@code{make install} -- @code{.la} files are never needed, to the extent
+that most GNU/Linux distributions remove them entirely. It is
+sufficient to just load the @code{.so} (or @code{.dll} and so on) files,
+which are always located in the same directory as the @code{.la} files.
+
+But for uninstalled dynamic libraries, like those in a build tree, the
+situation is a bit of a mess. If you have a project that uses libtool
+to build libraries -- which is the case for Guile, and for most projects
+using autotools -- and you build @file{foo.so} in directory @file{D},
+libtool will put @file{foo.la} in @file{D}, but @file{foo.so} gets put
+into @file{D/.libs}.
+
+Users were mostly oblivious to this situation, as @code{libltdl} had
+special logic to be able to read the @code{.la} file to know where to
+find the @code{.so}, even from an uninstalled build tree, preventing the
+existence of @file{.libs} from leaking out to the user.
+
+We don't use libltdl now, essentially for flexibility and
+error-reporting reasons. But, to keep this old use-case working, if
+@var{search-ltdl-library-path?} is true, we add each entry of
+@code{LTDL_LIBRARY_PATH} to the default extensions load path,
+additionally adding the @file{.libs} subdirextories for each entry, in
+case there are @file{.so} files there instead of alongside the
+@file{.la} files.
+@item GUILE_SYSTEM_EXTENSIONS_PATH
+The last path in Guile's search path belongs to Guile itself, and
+defaults to the libdir and the extensiondir, in that order. For
+example, if you install to @file{/opt/guile}, these would probably be
+@file{/opt/guile/lib} and
+@code{/opt/guile/lib/guile/@value{EFFECTIVE-VERSION}/extensions},
+respectively. @xref{Parallel Installations}, for more details on
+@code{extensionsdir}.
+@end table
+
+Finally, if no library is found in the search path, and if @var{library}
+is not absolute and does not include directory separators, and if
+@var{search-system-paths?} is true, the operating system may have its
+own logic for where to locate @var{library}. For example, on GNU, there
+will be a default set of paths (often @file{/usr/lib} and @file{/lib},
+though it depends on the system), and the @code{LD_LIBRARY_PATH}
+environment variable can add additional paths. Other operating systems
+have other conventions.
+
+Falling back to the operating system for search is usually not a great
+thing; it is a recipe for making programs that work on one machine but
+not on others. Still, when wrapping system libraries, it can be the
+only way to get things working at all.
+
+If @var{lazy?} is true (the default), Guile will request the operating
+system to resolve symbols used by the loaded library as they are first
+used. If @var{global?} is true, symbols defined by the loaded library
+will be available when other modules need to resolve symbols; the
+default is @code{#f}, which keeps symbols local.
@end deffn
-Guile has static support for calling functions with no arguments,
-@code{dynamic-call}.
-
-@deffn {Scheme Procedure} dynamic-call func dobj
-@deffnx {C Function} scm_dynamic_call (func, dobj)
-Call the C function indicated by @var{func} and @var{dobj}.
-The function is passed no arguments and its return value is
-ignored. When @var{function} is something returned by
-@code{dynamic-func}, call that function and ignore @var{dobj}.
-When @var{func} is a string , look it up in @var{dynobj}; this
-is equivalent to
-@smallexample
-(dynamic-call (dynamic-func @var{func} @var{dobj}) #f)
-@end smallexample
+The environment variables mentioned above are parsed when the
+foreign-library module is first loaded and bound to parameters. Null
+path components, for example the three components of
+@env{GUILE_SYSTEM_EXTENSIONS_PATH="::"}, are ignored.
+
+@deffn {Scheme Parameter} guile-extensions-path
+@deffnx {Scheme Parameter} ltdl-library-path
+@deffnx {Scheme Parameter} guile-system-extensions-path
+Parameters whose initial values are taken from
+@env{GUILE_EXTENSIONS_PATH}, @env{LTDL_LIBRARY_PATH}, and
+@env{GUILE_SYSTEM_EXTENSIONS_PATH}, respectively. @xref{Parameters}.
+The current values of these parameters are used when building the search
+path when @code{load-foreign-library} is called, unless the caller
+explicitly passes a @code{#:search-path} argument.
@end deffn
-@code{dynamic-call} is not very powerful. It is mostly intended to be
-used for calling specially written initialization functions that will
-then add new primitives to Guile. For example, we do not expect that you
-will dynamically link @file{libX11} with @code{dynamic-link} and then
-construct a beautiful graphical user interface just by using
-@code{dynamic-call}. Instead, the usual way would be to write a special
-Guile-to-X11 glue library that has intimate knowledge about both Guile
-and X11 and does whatever is necessary to make them inter-operate
-smoothly. This glue library could then be dynamically linked into a
-vanilla Guile interpreter and activated by calling its initialization
-function. That function would add all the new types and primitives to
-the Guile interpreter that it has to offer.
-
-(There is actually another, better option: simply to create a
-@file{libX11} wrapper in Scheme via the dynamic FFI. @xref{Dynamic FFI},
-for more information.)
-
-Given some set of C extensions to Guile, the next logical step is to
-integrate these glue libraries into the module system of Guile so that
-you can load new primitives into a running system just as you can load
-new Scheme code.
-
-@deffn {Scheme Procedure} load-extension lib init
-@deffnx {C Function} scm_load_extension (lib, init)
-Load and initialize the extension designated by LIB and INIT.
-When there is no pre-registered function for LIB/INIT, this is
-equivalent to
-
-@lisp
-(dynamic-call INIT (dynamic-link LIB))
-@end lisp
-
-When there is a pre-registered function, that function is called
-instead.
-
-Normally, there is no pre-registered function. This option exists
-only for situations where dynamic linking is unavailable or unwanted.
-In that case, you would statically link your program with the desired
-library, and register its init function right after Guile has been
-initialized.
-
-As for @code{dynamic-link}, @var{lib} should not contain any suffix such
-as @code{.so} (@pxref{Foreign Libraries, dynamic-link}). It
-should also not contain any directory components. Libraries that
-implement Guile Extensions should be put into the normal locations for
-shared libraries. We recommend to use the naming convention
-@file{libguile-bla-blum} for a extension related to a module @code{(bla
-blum)}.
-
-The normal way for a extension to be used is to write a small Scheme
-file that defines a module, and to load the extension into this
-module. When the module is auto-loaded, the extension is loaded as
-well. For example,
-
-@lisp
-(define-module (bla blum))
-
-(load-extension "libguile-bla-blum" "bla_init_blum")
-@end lisp
+@deffn {Scheme Procedure} foreign-library? obj
+Return @code{#t} if @var{obj} is a foreign library, or @code{#f}
+otherwise.
@end deffn
-@node C Extensions
-@subsection C Extensions
-The most interesting application of dynamically linked libraries is
-probably to use them for providing @emph{compiled code modules} to
-Scheme programs. As much fun as programming in Scheme is, every now and
-then comes the need to write some low-level C stuff to make Scheme even
-more fun.
+@node Foreign Extensions
+@subsection Foreign Extensions
-Not only can you put these new primitives into their own module (see the
-previous section), you can even put them into a shared library that is
-only then linked to your running Guile image when it is actually
-needed.
+One way to use shared libraries is to extend Guile. Such loadable
+modules generally define one distinguished initialization function that,
+when called, will use the @code{libguile} API to define procedures in
+the current module.
-An example will hopefully make everything clear. Suppose we want to
-make the Bessel functions of the C library available to Scheme in the
-module @samp{(math bessel)}. First we need to write the appropriate
-glue code to convert the arguments and return values of the functions
-from Scheme to C and back. Additionally, we need a function that will
-add them to the set of Guile primitives. Because this is just an
-example, we will only implement this for the @code{j0} function.
+Concretely, you might extend Guile with an implementation of the Bessel
+function, @code{j0}:
@smallexample
#include <math.h>
@@ -260,211 +223,222 @@ j0_wrapper (SCM x)
@}
void
-init_math_bessel ()
+init_math_bessel (void)
@{
scm_c_define_gsubr ("j0", 1, 0, 0, j0_wrapper);
@}
@end smallexample
-We can already try to bring this into action by manually calling the low
-level functions for performing dynamic linking. The C source file needs
-to be compiled into a shared library. Here is how to do it on
-GNU/Linux, please refer to the @code{libtool} documentation for how to
-create dynamically linkable libraries portably.
+The C source file would then need to be compiled into a shared library.
+On GNU/Linux, the compiler invocation might look like this:
@smallexample
-gcc -shared -o libbessel.so -fPIC bessel.c
+gcc -shared -o bessel.so -fPIC bessel.c
@end smallexample
-Now fire up Guile:
+A good default place to put shared libraries that extend Guile is into
+the extensions dir. From the command line or a build script, invoke
+@code{pkg-config --variable=extensionsdir
+guile-@value{EFFECTIVE-VERSION}} to print the extensions dir.
+@xref{Parallel Installations}, for more details.
+
+Guile can load up @code{bessel.so} via @code{load-extension}.
+
+@deffn {Scheme Procedure} load-extension lib init
+@deffnx {C Function} scm_load_extension (lib, init)
+Load and initialize the extension designated by LIB and INIT.
+@end deffn
+
+The normal way for a extension to be used is to write a small Scheme
+file that defines a module, and to load the extension into this
+module. When the module is auto-loaded, the extension is loaded as
+well. For example:
@lisp
-(define bessel-lib (dynamic-link "./libbessel.so"))
-(dynamic-call "init_math_bessel" bessel-lib)
-(j0 2)
-@result{} 0.223890779141236
+(define-module (math bessel)
+ #:export (j0))
+
+(load-extension "bessel" "init_math_bessel")
@end lisp
-The filename @file{./libbessel.so} should be pointing to the shared
-library produced with the @code{gcc} command above, of course. The
-second line of the Guile interaction will call the
-@code{init_math_bessel} function which in turn will register the C
-function @code{j0_wrapper} with the Guile interpreter under the name
-@code{j0}. This function becomes immediately available and we can call
-it from Scheme.
+This @code{load-extension} invocation loads the @code{bessel} library
+via @code{(load-foreign-library "bessel")}, then looks up the
+@code{init_math_bessel} symbol in the library, treating it as a function
+of no arguments, and calls that function.
-Fun, isn't it? But we are only half way there. This is what
-@code{apropos} has to say about @code{j0}:
+If you decide to put your extension outside the default search path for
+@code{load-foreign-library}, probably you should adapt the Scheme module
+to specify its absolute path. For example, if you use @code{automake}
+to build your extension and place it in @code{$(pkglibdir)}, you might
+define a build-parameters module that gets created by the build system:
-@smallexample
-(apropos "j0")
-@print{} (guile-user): j0 #<primitive-procedure j0>
-@end smallexample
+@example
+(define-module (math config)
+ #:export (extensiondir))
+(define extensiondir "PKGLIBDIR")
+@end example
+
+This file would be @code{config.scm.in}. You would define a @code{make}
+rule to substitute in the absolute installed file name:
+
+@example
+config.scm: config.scm.in
+ sed 's|PKGLIBDIR|$(pkglibdir)|' <$< >$@
+@end example
+
+Then your @code{(math bessel)} would import @code{(math config)}, then
+@code{(load-extension (in-vicinity extensiondir "bessel")
+"init_math_bessel")}.
-As you can see, @code{j0} is contained in the root module, where all
-the other Guile primitives like @code{display}, etc live. In general,
-a primitive is put into whatever module is the @dfn{current module} at
-the time @code{scm_c_define_gsubr} is called.
+An alternate approach would be to rebind the
+@code{guile-extensions-path} parameter, or its corresponding environment
+variable, but note that changing those parameters applies to other users
+of @code{load-foreign-library} as well.
-A compiled module should have a specially named @dfn{module init
-function}. Guile knows about this special name and will call that
-function automatically after having linked in the shared library. For
-our example, we replace @code{init_math_bessel} with the following code in
-@file{bessel.c}:
+Note that the new primitives that the extension adds to Guile with
+@code{scm_c_define_gsubr} (@pxref{Primitive Procedures}) or with any of
+the other mechanisms are placed into the module that is current when the
+@code{scm_c_define_gsubr} is executed, so to be clear about what goes
+vwhere it's best to include the @code{load-extension} in a module, as
+above. Alternately, the C code can use @code{scm_c_define_module} to
+specify which module is being created:
@smallexample
-void
-init_math_bessel (void *unused)
+static void
+do_init (void *unused)
@{
scm_c_define_gsubr ("j0", 1, 0, 0, j0_wrapper);
scm_c_export ("j0", NULL);
@}
void
-scm_init_math_bessel_module ()
+init_math_bessel ()
@{
- scm_c_define_module ("math bessel", init_math_bessel, NULL);
+ scm_c_define_module ("math bessel", do_init, NULL);
@}
@end smallexample
-The general pattern for the name of a module init function is:
-@samp{scm_init_}, followed by the name of the module where the
-individual hierarchical components are concatenated with underscores,
-followed by @samp{_module}.
+And yet... if what we want is just the @code{j0} function, it seems like
+a lot of ceremony to have to compile a Guile-specific wrapper library
+complete with an initialization function and wraper module to allow
+Guile users to call it. There is another way, but to get there, we have
+to talk about function pointers and function types first. @xref{Foreign
+Functions}, to skip to the good parts.
-After @file{libbessel.so} has been rebuilt, we need to place the shared
-library into the right place.
-Once the module has been correctly installed, it should be possible to
-use it like this:
-
-@smallexample
-guile> (load-extension "./libbessel.so" "scm_init_math_bessel_module")
-guile> (use-modules (math bessel))
-guile> (j0 2)
-0.223890779141236
-guile> (apropos "j0")
-@print{} (math bessel): j0 #<primitive-procedure j0>
-@end smallexample
-
-That's it!
+@node Foreign Pointers
+@subsection Foreign Pointers
+Foreign libraries are essentially key-value mappings, where the keys are
+names of definitions and the values are the addresses of those
+definitions. To look up the address of a definition, use
+@code{foreign-library-pointer} from the @code{(system foreign-library)}
+module.
-@node Modules and Extensions
-@subsection Modules and Extensions
+@deffn {Scheme Procedure} foreign-library-pointer lib name
+Return a ``wrapped pointer'' for the symbol @var{name} in the shared
+object referred to by @var{lib}. The returned pointer points to a C
+object.
-The new primitives that you add to Guile with @code{scm_c_define_gsubr}
-(@pxref{Primitive Procedures}) or with any of the other mechanisms are
-placed into the module that is current when the
-@code{scm_c_define_gsubr} is executed. Extensions loaded from the REPL,
-for example, will be placed into the @code{(guile-user)} module, if the
-REPL module was not changed.
+As a convenience, if @var{lib} is not a foreign library, it will be
+passed to @code{load-foreign-library}.
+@end deffn
-To define C primitives within a specific module, the simplest way is:
+If we continue with the @code{bessel.so} example from before, we can get
+the address of the @code{init_math_bessel} function via:
@example
-(define-module (foo bar))
-(load-extension "foobar-c-code" "foo_bar_init")
+(use-modules (system foreign-library))
+(define init (foreign-library-pointer "bessel" "init_math_bessel"))
+init
+@result{} #<pointer 0x7fb35b1b4688>
@end example
-@cindex extensiondir
-When loaded with @code{(use-modules (foo bar))}, the
-@code{load-extension} call looks for the @file{foobar-c-code.so} (etc)
-object file in Guile's @code{extensiondir}, which is usually a
-subdirectory of the @code{libdir}. For example, if your libdir is
-@file{/usr/lib}, the @code{extensiondir} for the Guile
@value{EFFECTIVE-VERSION}.@var{x}
-series will be @file{/usr/lib/guile/@value{EFFECTIVE-VERSION}/}.
-
-The extension path includes the major and minor version of Guile (the
-``effective version''), because Guile guarantees compatibility within a
-given effective version. This allows you to install different versions
-of the same extension for different versions of Guile.
-
-If the extension is not found in the @code{extensiondir}, Guile will
-also search the standard system locations, such as @file{/usr/lib} or
-@file{/usr/local/lib}. It is preferable, however, to keep your extension
-out of the system library path, to prevent unintended interference with
-other dynamically-linked C libraries.
-
-If someone installs your module to a non-standard location then the
-object file won't be found. You can address this by inserting the
-install location in the @file{foo/bar.scm} file. This is convenient
-for the user and also guarantees the intended object is read, even if
-stray older or newer versions are in the loader's path.
-
-The usual way to specify an install location is with a @code{prefix}
-at the configure stage, for instance @samp{./configure prefix=/opt}
-results in library files as say @file{/opt/lib/foobar-c-code.so}.
-When using Autoconf (@pxref{Top, , Introduction, autoconf, The GNU
-Autoconf Manual}), the library location is in a @code{libdir}
-variable. Its value is intended to be expanded by @command{make}, and
-can by substituted into a source file like @file{foo.scm.in}
+A value returned by @code{foreign-library-pointer} is a Scheme wrapper
+for a C pointer. Pointers are a data type in Guile that is disjoint
+from all other types. The next section discusses ways to dereference
+pointers, but before then we describe the usual type predicates and so
+on.
+
+Note that the rest of the interfaces in this section are part of the
+@code{(system foreign)} library:
@example
-(define-module (foo bar))
-(load-extension "XXextensiondirXX/foobar-c-code" "foo_bar_init")
+(use-modules (system foreign))
@end example
-@noindent
-with the following in a @file{Makefile}, using @command{sed}
-(@pxref{Top, , Introduction, sed, SED, A Stream Editor}),
+@deffn {Scheme Procedure} pointer-address pointer
+@deffnx {C Function} scm_pointer_address (pointer)
+Return the numerical value of @var{pointer}.
@example
-foo.scm: foo.scm.in
- sed 's|XXextensiondirXX|$(libdir)/guile/@value{EFFECTIVE-VERSION}|'
<foo.scm.in >foo.scm
+(pointer-address init)
+@result{} 139984413364296 ; YMMV
@end example
+@end deffn
-The actual pattern @code{XXextensiondirXX} is arbitrary, it's only something
-which doesn't otherwise occur. If several modules need the value, it
-can be easier to create one @file{foo/config.scm} with a define of the
-@code{extensiondir} location, and use that as required.
+@deffn {Scheme Procedure} make-pointer address [finalizer]
+Return a foreign pointer object pointing to @var{address}. If
+@var{finalizer} is passed, it should be a pointer to a one-argument C
+function that will be called when the pointer object becomes
+unreachable.
+@end deffn
-@example
-(define-module (foo config))
-(define-public foo-config-extensiondir "XXextensiondirXX"")
-@end example
+@deffn {Scheme Procedure} pointer? obj
+Return @code{#t} if @var{obj} is a pointer object, or @code{#f}
+otherwise.
+@end deffn
-Such a file might have other locations too, for instance a data
-directory for auxiliary files, or @code{localedir} if the module has
-its own @code{gettext} message catalogue
-(@pxref{Internationalization}).
+@defvr {Scheme Variable} %null-pointer
+A foreign pointer whose value is 0.
+@end defvr
-It will be noted all of the above requires that the Scheme code to be
-found in @code{%load-path} (@pxref{Load Paths}). Presently it's left up
-to the system administrator or each user to augment that path when
-installing Guile modules in non-default locations. But having reached
-the Scheme code, that code should take care of hitting any of its own
-private files etc.
+@deffn {Scheme Procedure} null-pointer? pointer
+Return @code{#t} if @var{pointer} is the null pointer, @code{#f} otherwise.
+@end deffn
+For the purpose of passing SCM values directly to foreign functions, and
+allowing them to return SCM values, Guile also supports some unsafe
+casting operators.
-@node Foreign Pointers
-@subsection Foreign Pointers
+@deffn {Scheme Procedure} scm->pointer scm
+Return a foreign pointer object with the @code{object-address}
+of @var{scm}.
+@end deffn
-The previous sections have shown how Guile can be extended at runtime by
-loading compiled C extensions. This approach is all well and good, but
-wouldn't it be nice if we didn't have to write any C at all? This
-section takes up the problem of accessing C values from Scheme, and the
-next discusses C functions.
+@deffn {Scheme Procedure} pointer->scm pointer
+Unsafely cast @var{pointer} to a Scheme object.
+Cross your fingers!
+@end deffn
-@menu
-* Foreign Types:: Expressing C types in Scheme.
-* Foreign Variables:: Pointers to C symbols.
-* Void Pointers and Byte Access:: Pointers into the ether.
-* Foreign Structs:: Packing and unpacking structs.
-@end menu
+Sometimes you want to give C extensions access to the dynamic FFI. At
+that point, the names get confusing, because ``pointer'' can refer to a
+@code{SCM} object that wraps a pointer, or to a @code{void*} value. We
+will try to use ``pointer object'' to refer to Scheme objects, and
+``pointer value'' to refer to @code{void *} values.
-@node Foreign Types
-@subsubsection Foreign Types
+@deftypefn {C Function} SCM scm_from_pointer (void *ptr, void (*finalizer)
(void*))
+Create a pointer object from a pointer value.
-The first impedance mismatch that one sees between C and Scheme is that
-in C, the storage locations (variables) are typed, but in Scheme types
-are associated with values, not variables. @xref{Values and Variables}.
+If @var{finalizer} is non-null, Guile arranges to call it on the pointer
+value at some point after the pointer object becomes collectable.
+@end deftypefn
-So when describing a C function or a C structure so that it can be
-accessed from Scheme, the data types of the parameters or fields must be
-passed explicitly.
+@deftypefn {C Function} void* scm_to_pointer (SCM obj)
+Unpack the pointer value from a pointer object.
+@end deftypefn
-These ``C type values'' may be constructed using the constants and
+@node Foreign Types
+@subsection Foreign Types
+
+From Scheme's perspective, foreign pointers are shards of chaos. The
+user can create a foreign pointer for any address, and do with it what
+they will. The only thing that lends a sense of order to the whole is a
+shared hallucination that certain storage locations have certain types.
+When making Scheme wrappers for foreign interfaces, we hide the madness
+by explicitly representing the the data types of parameters and fields.
+
+These ``foreign type values'' may be constructed using the constants and
procedures from the @code{(system foreign)} module, which may be loaded
like this:
@@ -473,7 +447,7 @@ like this:
@end example
@code{(system foreign)} exports a number of values expressing the basic
-C types:
+C types.
@defvr {Scheme Variable} int8
@defvrx {Scheme Variable} uint8
@@ -490,7 +464,7 @@ signednesses.
@end defvr
In addition there are some convenience bindings for indicating types of
-platform-dependent size:
+platform-dependent size.
@defvr {Scheme Variable} int
@defvrx {Scheme Variable} unsigned-int
@@ -517,101 +491,81 @@ In addition, the symbol @code{*} is used by convention
to denote pointer
types. Procedures detailed in the following sections, such as
@code{pointer->procedure}, accept it as a type descriptor.
-@node Foreign Variables
-@subsubsection Foreign Variables
-
-Pointers to variables in the current address space may be looked up
-dynamically using @code{dynamic-pointer}.
-
-@deffn {Scheme Procedure} dynamic-pointer name dobj
-@deffnx {C Function} scm_dynamic_pointer (name, dobj)
-Return a ``wrapped pointer'' for the symbol @var{name} in the shared
-object referred to by @var{dobj}. The returned pointer points to a C
-object.
-
-Regardless whether your C compiler prepends an underscore @samp{_} to the
global
-names in a program, you should @strong{not} include this underscore in
-@var{name} since it will be added automatically when necessary.
-@end deffn
+@node Foreign Functions
+@subsection Foreign Functions
-For example, currently Guile has a variable, @code{scm_numptob}, as part
-of its API. It is declared as a C @code{long}. So, to create a handle
-pointing to that foreign value, we do:
+The most natural thing to do with a dynamic library is to grovel around
+in it for a function pointer: a @dfn{foreign function}. Load the
+@code{(system foreign)} module to use these Scheme interfaces.
@example
(use-modules (system foreign))
-(define numptob (dynamic-pointer "scm_numptob" (dynamic-link)))
-numptob
-@result{} #<pointer 0x7fb35b1b4688>
@end example
-(The next section discusses ways to dereference pointers.)
-
-A value returned by @code{dynamic-pointer} is a Scheme wrapper for a C
-pointer.
+@deffn {Scheme Procedure} pointer->procedure return_type func_ptr arg_types @
+ [#:return-errno?=#f]
+@deffnx {C Function} scm_pointer_to_procedure (return_type, func_ptr,
arg_types)
+@deffnx {C Function} scm_pointer_to_procedure_with_errno (return_type,
func_ptr, arg_types)
-@deffn {Scheme Procedure} pointer-address pointer
-@deffnx {C Function} scm_pointer_address (pointer)
-Return the numerical value of @var{pointer}.
+Make a foreign function.
-@example
-(pointer-address numptob)
-@result{} 139984413364296 ; YMMV
-@end example
-@end deffn
+Given the foreign void pointer @var{func_ptr}, its argument and
+return types @var{arg_types} and @var{return_type}, return a
+procedure that will pass arguments to the foreign function
+and return appropriate values.
-@deffn {Scheme Procedure} make-pointer address [finalizer]
-Return a foreign pointer object pointing to @var{address}. If
-@var{finalizer} is passed, it should be a pointer to a one-argument C
-function that will be called when the pointer object becomes
-unreachable.
-@end deffn
+@var{arg_types} should be a list of foreign types.
+@code{return_type} should be a foreign type. @xref{Foreign Types}, for
+more information on foreign types.
-@deffn {Scheme Procedure} pointer? obj
-Return @code{#t} if @var{obj} is a pointer object, @code{#f} otherwise.
+If @var{return-errno?} is true, or when calling
+@code{scm_pointer_to_procedure_with_errno}, the returned procedure will
+return two values, with @code{errno} as the second value.
@end deffn
-@defvr {Scheme Variable} %null-pointer
-A foreign pointer whose value is 0.
-@end defvr
+Finally, in @code{(system foreign-library)} there is a convenient
+wrapper function, joining together @code{foreign-libary-pointer} and
+@code{procedure->pointer}:
-@deffn {Scheme Procedure} null-pointer? pointer
-Return @code{#t} if @var{pointer} is the null pointer, @code{#f} otherwise.
-@end deffn
+@deffn {Scheme Procedure} foreign-library-function lib name @
+ [#:return-type=void] [#:arg-types='()] [#:return-errno?=#f]
+Load the address of @var{name} from @var{lib}, and treat it as a
+function taking arguments @var{arg-types} and returning
+@var{return-type}, optionally also with errno.
-For the purpose of passing SCM values directly to foreign functions, and
-allowing them to return SCM values, Guile also supports some unsafe
-casting operators.
-
-@deffn {Scheme Procedure} scm->pointer scm
-Return a foreign pointer object with the @code{object-address}
-of @var{scm}.
+An invocation of @code{foreign-library-function} is entirely equivalent
+to:
+@example
+(pointer->procedure @var{return-type}
+ (foreign-library-pointer @var{lib} @var{name})
+ @var{arg-types}
+ #:return-errno? @var{return-errno?}).
+@end example
@end deffn
-@deffn {Scheme Procedure} pointer->scm pointer
-Unsafely cast @var{pointer} to a Scheme object.
-Cross your fingers!
-@end deffn
+Pulling all this together, here is a better definition of @code{(math
+bessel)}:
-Sometimes you want to give C extensions access to the dynamic FFI. At
-that point, the names get confusing, because ``pointer'' can refer to a
-@code{SCM} object that wraps a pointer, or to a @code{void*} value. We
-will try to use ``pointer object'' to refer to Scheme objects, and
-``pointer value'' to refer to @code{void *} values.
+@example
+(define-module (math bessel)
+ #:use-module (system foreign)
+ #:use-module (system foreign-library)
+ #:export (j0))
-@deftypefn {C Function} SCM scm_from_pointer (void *ptr, void (*finalizer)
(void*))
-Create a pointer object from a pointer value.
+(define j0
+ (foreign-library-function "libm" "j0"
+ #:return-type double
+ #:arg-types (list double)))
+@end example
-If @var{finalizer} is non-null, Guile arranges to call it on the pointer
-value at some point after the pointer object becomes collectable.
-@end deftypefn
+That's it! No C at all.
-@deftypefn {C Function} void* scm_to_pointer (SCM obj)
-Unpack the pointer value from a pointer object.
-@end deftypefn
+Before going on to more detailed examples, the next two sections discuss
+how to deal with data that is more complex than, say, @code{int8}.
+@xref{More Foreign Functions}, to continue with foreign function examples.
@node Void Pointers and Byte Access
-@subsubsection Void Pointers and Byte Access
+@subsection Void Pointers and Byte Access
Wrapped pointers are untyped, so they are essentially equivalent to C
@code{void} pointers. As in C, the memory region pointed to by a
@@ -620,6 +574,12 @@ pointer can be accessed at the byte level. This is
achieved using
module contains procedures that can be used to convert byte sequences to
Scheme objects such as strings, floating point numbers, or integers.
+Load the @code{(system foreign)} module to use these Scheme interfaces.
+
+@example
+(use-modules (system foreign))
+@end example
+
@deffn {Scheme Procedure} pointer->bytevector pointer len [offset [uvec_type]]
@deffnx {C Function} scm_pointer_to_bytevector (pointer, len, offset,
uvec_type)
Return a bytevector aliasing the @var{len} bytes pointed to by
@@ -709,19 +669,17 @@ pointers to manipulate them. We could write:
(define grab-bottle
;; Wrapper for `bottle_t *grab (void)'.
- (let ((grab (pointer->procedure '*
- (dynamic-func "grab_bottle" libbottle)
- '())))
+ (let ((grab (foreign-library-function libbottle "grab_bottle"
+ #:return-type '*)))
(lambda ()
"Return a new bottle."
(wrap-bottle (grab)))))
(define bottle-contents
;; Wrapper for `const char *bottle_contents (bottle_t *)'.
- (let ((contents (pointer->procedure '*
- (dynamic-func "bottle_contents"
- libbottle)
- '(*))))
+ (let ((contents (foreign-library-function libbottle "bottle_contents"
+ #:return-type '*
+ #:arg-types '(*))))
(lambda (b)
"Return the contents of B."
(pointer->string (contents (unwrap-bottle b))))))
@@ -736,12 +694,16 @@ In this example, @code{grab-bottle} is guaranteed to
return a genuine
@code{bottle} object.
@end deffn
-Going back to the @code{scm_numptob} example above, here is how we can
-read its value as a C @code{long} integer:
+As another example, currently Guile has a variable, @code{scm_numptob},
+as part of its API. It is declared as a C @code{long}. So, to read its
+value, we can do:
@example
+(use-modules (system foreign))
(use-modules (rnrs bytevectors))
-
+(define numptob
+ (foreign-library-pointer #f "scm_numptob"))
+numptob
(bytevector-uint-ref (pointer->bytevector numptob (sizeof long))
0 (native-endianness)
(sizeof long))
@@ -756,13 +718,17 @@ crash your program, simply accessing the data pointed to
by a dangling
pointer or similar can prove equally disastrous.
@node Foreign Structs
-@subsubsection Foreign Structs
+@subsection Foreign Structs
Finally, one last note on foreign values before moving on to actually
calling foreign functions. Sometimes you need to deal with C structs,
which requires interpreting each element of the struct according to the
-its type, offset, and alignment. Guile has some primitives to support
-this.
+its type, offset, and alignment. The @code{(system foreign)} module has
+some primitives to support this.
+
+@example
+(use-modules (system foreign))
+@end example
@deffn {Scheme Procedure} sizeof type
@deffnx {C Function} scm_sizeof (type)
@@ -818,67 +784,20 @@ and @code{pointer->bytevector} routines, one can create
and parse
tightly packed structs and unions by hand. See the code for
@code{(system foreign)} for details.
+@node More Foreign Functions
+@subsection More Foreign Functions
-@node Dynamic FFI
-@subsection Dynamic FFI
-
-Of course, the land of C is not all nouns and no verbs: there are
-functions too, and Guile allows you to call them.
-
-@deffn {Scheme Procedure} pointer->procedure return_type func_ptr arg_types @
- [#:return-errno?=#f]
-@deffnx {C Function} scm_pointer_to_procedure (return_type, func_ptr,
arg_types)
-@deffnx {C Function} scm_pointer_to_procedure_with_errno (return_type,
func_ptr, arg_types)
-
-Make a foreign function.
-
-Given the foreign void pointer @var{func_ptr}, its argument and
-return types @var{arg_types} and @var{return_type}, return a
-procedure that will pass arguments to the foreign function
-and return appropriate values.
-
-@var{arg_types} should be a list of foreign types.
-@code{return_type} should be a foreign type. @xref{Foreign Types}, for
-more information on foreign types.
-
-If @var{return-errno?} is true, or when calling
-@code{scm_pointer_to_procedure_with_errno}, the returned procedure will
-return two values, with @code{errno} as the second value.
-@end deffn
-
-Here is a better definition of @code{(math bessel)}:
-
-@example
-(define-module (math bessel)
- #:use-module (system foreign)
- #:export (j0))
-
-(define libm (dynamic-link "libm"))
-
-(define j0
- (pointer->procedure double
- (dynamic-func "j0" libm)
- (list double)))
-@end example
-
-That's it! No C at all.
-
-Numeric arguments and return values from foreign functions are
-represented as Scheme values. For example, @code{j0} in the above
-example takes a Scheme number as its argument, and returns a Scheme
-number.
-
-Pointers may be passed to and returned from foreign functions as well.
-In that case the type of the argument or return value should be the
-symbol @code{*}, indicating a pointer. For example, the following
+It is possible to pass pointers to foreign functions, and to return them
+as well. In that case the type of the argument or return value should
+be the symbol @code{*}, indicating a pointer. For example, the following
code makes @code{memcpy} available to Scheme:
@example
+(use-modules (system foreign))
(define memcpy
- (let ((this (dynamic-link)))
- (pointer->procedure '*
- (dynamic-func "memcpy" this)
- (list '* '* size_t))))
+ (foreign-library-function #f "memcpy"
+ #:return-type '*
+ #:arg-types (list '* '* size_t)))
@end example
To invoke @code{memcpy}, one must pass it foreign pointers:
@@ -914,10 +833,9 @@ by the foreign pointer is mutated in place.
;; assuming fields are of type "long"
(define gettimeofday
- (let ((f (pointer->procedure
- int
- (dynamic-func "gettimeofday" (dynamic-link))
- (list '* '*)))
+ (let ((f (foreign-library-function #f "gettimeofday"
+ #:return-type int
+ #:arg-types (list '* '*)))
(tv-type (list long long)))
(lambda ()
(let* ((timeval (make-c-struct tv-type (list 0 0)))
@@ -955,10 +873,8 @@ function can be made accessible to Scheme (@pxref{Array
Sort Function,
@example
(define qsort!
- (let ((qsort (pointer->procedure void
- (dynamic-func "qsort"
- (dynamic-link))
- (list '* size_t size_t '*))))
+ (let ((qsort (foreign-library-function
+ #f "qsort" #:arg-types (list '* size_t size_t '*))))
(lambda (bv compare)
;; Sort bytevector BV in-place according to comparison
;; procedure COMPARE.
diff --git a/doc/ref/guile.texi b/doc/ref/guile.texi
index 9f3fe2d..660b1ae 100644
--- a/doc/ref/guile.texi
+++ b/doc/ref/guile.texi
@@ -13,7 +13,7 @@
@copying
This manual documents Guile version @value{VERSION}.
-Copyright (C) 1996-1997, 2000-2005, 2009-2020 Free Software Foundation,
+Copyright (C) 1996-1997, 2000-2005, 2009-2021 Free Software Foundation,
Inc.
Permission is granted to copy, distribute and/or modify this document
@@ -299,8 +299,6 @@ available through both Scheme and C interfaces.
* Initialization:: Initializing Guile.
* Snarfing Macros:: Macros for snarfing initialization actions.
* Data Types:: Representing values in Guile.
-* Foreign Objects:: Defining new data types in C.
-* Smobs:: Use foreign objects instead.
* Procedures:: Procedures.
* Macros:: Extending the syntax of Scheme.
* Utility Functions:: General utility functions.
@@ -314,6 +312,8 @@ available through both Scheme and C interfaces.
* Memory Management:: Memory management and garbage collection.
* Modules:: Designing reusable code libraries.
* Foreign Function Interface:: Interacting with C procedures and data.
+* Foreign Objects:: Defining new data types in C.
+* Smobs:: Use foreign objects instead.
* Scheduling:: Threads, mutexes, asyncs and dynamic roots.
* Options and Config:: Configuration, features and runtime options.
* Other Languages:: Emacs Lisp, ECMAScript, and more.
@@ -328,8 +328,6 @@ available through both Scheme and C interfaces.
@include api-init.texi
@include api-snarf.texi
@include api-data.texi
-@include api-foreign-objects.texi
-@include api-smobs.texi
@include api-procedures.texi
@include api-macros.texi
@include api-utility.texi
@@ -343,6 +341,8 @@ available through both Scheme and C interfaces.
@include api-memory.texi
@include api-modules.texi
@include api-foreign.texi
+@include api-foreign-objects.texi
+@include api-smobs.texi
@include api-scheduling.texi
@c object orientation support here
@include api-options.texi
diff --git a/doc/ref/libguile-parallel.texi b/doc/ref/libguile-parallel.texi
index 75fcd88..a3779a2 100644
--- a/doc/ref/libguile-parallel.texi
+++ b/doc/ref/libguile-parallel.texi
@@ -1,7 +1,7 @@
@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
@c Copyright (C) 1996, 1997, 2000, 2001, 2002, 2003, 2004, 2005, 2010, 2011,
-@c 2013-2014 Free Software Foundation, Inc.
+@c 2013-2014, 2021 Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@node Parallel Installations
@@ -52,8 +52,8 @@ how to use it from Autoconf.
@item extensiondir
@cindex @code{extensiondir}
The default directory where Guile looks for extensions---i.e., shared
-libraries providing additional features (@pxref{Modules and
-Extensions}). Run @command{pkg-config guile-@value{EFFECTIVE-VERSION}
+libraries providing additional features (@pxref{Foreign Extensions}).
+Run @command{pkg-config guile-@value{EFFECTIVE-VERSION}
--variable=extensiondir} to see its value.
@item guile
diff --git a/doc/ref/tour.texi b/doc/ref/tour.texi
index 0cac96c..c0ecb16 100644
--- a/doc/ref/tour.texi
+++ b/doc/ref/tour.texi
@@ -1,7 +1,7 @@
@c -*-texinfo-*-
@c This is part of the GNU Guile Reference Manual.
@c Copyright (C) 1996, 1997, 2000, 2001, 2002, 2003, 2004, 2006, 2010, 2011,
-@c 2012 Free Software Foundation, Inc.
+@c 2012, 2021 Free Software Foundation, Inc.
@c See the file guile.texi for copying conditions.
@raisesections
@@ -280,7 +280,7 @@ scheme@@(guile-user)> (j0 2)
$1 = 0.223890779141236
@end smallexample
-@xref{Modules and Extensions}, for more information.
+@xref{Foreign Extensions}, for more information.
@lowersections
diff --git a/libguile/deprecated.c b/libguile/deprecated.c
index fcc4e83..e4909df 100644
--- a/libguile/deprecated.c
+++ b/libguile/deprecated.c
@@ -1,4 +1,4 @@
-/* Copyright 2003-2004,2006,2008-2018,2020
+/* Copyright 2003-2004,2006,2008-2018,2020,2021
Free Software Foundation, Inc.
This file is part of Guile.
@@ -31,7 +31,9 @@
#include "boolean.h"
#include "bitvectors.h"
#include "deprecation.h"
+#include "dynl.h"
#include "eval.h"
+#include "foreign.h"
#include "gc.h"
#include "gsubr.h"
#include "modules.h"
@@ -601,6 +603,19 @@ scm_copy_tree (SCM obj)
+SCM_DEFINE (scm_dynamic_unlink, "dynamic-unlink", 1, 0, 0, (SCM obj), "")
+#define FUNC_NAME s_scm_dynamic_unlink
+{
+ scm_c_issue_deprecation_warning
+ ("scm_dynamic_unlink has no effect and is deprecated. Unloading "
+ "shared libraries is no longer supported.");
+ return SCM_UNSPECIFIED;
+}
+#undef FUNC_NAME
+
+
+
+
void
scm_i_init_deprecated ()
{
diff --git a/libguile/deprecated.h b/libguile/deprecated.h
index c95f919..c68decf 100644
--- a/libguile/deprecated.h
+++ b/libguile/deprecated.h
@@ -1,7 +1,7 @@
#ifndef SCM_DEPRECATED_H
#define SCM_DEPRECATED_H
-/* Copyright 2003-2007,2009-2018,2020
+/* Copyright 2003-2007,2009-2018,2020,2021
Free Software Foundation, Inc.
This file is part of Guile.
@@ -140,6 +140,8 @@ SCM_DEPRECATED SCM scm_make_srcprops (long line, int col,
SCM filename,
SCM_DEPRECATED SCM scm_copy_tree (SCM obj);
+SCM_DEPRECATED SCM scm_dynamic_unlink (SCM obj);
+
void scm_i_init_deprecated (void);
#endif
diff --git a/libguile/dynl.c b/libguile/dynl.c
index e9c03e9..8e1cc90 100644
--- a/libguile/dynl.c
+++ b/libguile/dynl.c
@@ -1,6 +1,6 @@
/* dynl.c - dynamic linking
- Copyright 1990-2003,2008-2011,2017-2018
+ Copyright 1990-2003,2008-2011,2017-2018,2021
Free Software Foundation, Inc.
This file is part of Guile.
@@ -28,369 +28,172 @@
# include <config.h>
#endif
-#include <alloca.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include <ltdl.h>
+#include <dlfcn.h>
+#include "boolean.h"
#include "deprecation.h"
-#include "dynwind.h"
+#include "eval.h"
+#include "extensions.h"
#include "foreign.h"
-#include "gc.h"
#include "gsubr.h"
-#include "keywords.h"
-#include "libpath.h"
#include "list.h"
-#include "ports.h"
-#include "smob.h"
+#include "modules.h"
+#include "numbers.h"
#include "strings.h"
#include "threads.h"
+#include "variable.h"
+#include "version.h"
#include "dynl.h"
-/* From the libtool manual: "Note that libltdl is not threadsafe,
- i.e. a multithreaded application has to use a mutex for libltdl.".
- Note: We initialize it as a recursive mutex below. */
-static scm_i_pthread_mutex_t ltdl_lock = SCM_I_PTHREAD_MUTEX_INITIALIZER;
-
-/* LT_PATH_SEP-separated extension library search path, searched last */
-static char *system_extensions_path;
-
-static void *
-sysdep_dynl_link (const char *fname, const char *subr)
-{
- lt_dlhandle handle;
-
- /* Try the literal filename first or, if NULL, the program itself */
- handle = lt_dlopen (fname);
- if (handle == NULL)
- {
- handle = lt_dlopenext (fname);
-
- if (handle == NULL
-#ifdef LT_DIRSEP_CHAR
- && strchr (fname, LT_DIRSEP_CHAR) == NULL
-#endif
- && strchr (fname, '/') == NULL)
- {
- /* FNAME contains no directory separators and was not in the
- usual library search paths, so now we search for it in
- SYSTEM_EXTENSIONS_PATH. */
- char *fname_attempt
- = scm_gc_malloc_pointerless (strlen (system_extensions_path)
- + strlen (fname) + 2,
- "dynl fname_attempt");
- char *path; /* remaining path to search */
- char *end; /* end of current path component */
- char *s;
-
- /* Iterate over the components of SYSTEM_EXTENSIONS_PATH */
- for (path = system_extensions_path;
- *path != '\0';
- path = (*end == '\0') ? end : (end + 1))
- {
- /* Find end of path component */
- end = strchr (path, LT_PATHSEP_CHAR);
- if (end == NULL)
- end = strchr (path, '\0');
-
- /* Skip empty path components */
- if (path == end)
- continue;
-
- /* Construct FNAME_ATTEMPT, starting with path component */
- s = fname_attempt;
- memcpy (s, path, end - path);
- s += end - path;
-
- /* Append directory separator, but avoid duplicates */
- if (s[-1] != '/'
-#ifdef LT_DIRSEP_CHAR
- && s[-1] != LT_DIRSEP_CHAR
-#endif
- )
- *s++ = '/';
-
- /* Finally, append FNAME (including null terminator) */
- strcpy (s, fname);
-
- /* Try to load it, and terminate the search if successful */
- handle = lt_dlopenext (fname_attempt);
- if (handle != NULL)
- break;
- }
- }
- }
-
- if (handle == NULL)
- {
- SCM fn;
- SCM msg;
-
- fn = fname != NULL ? scm_from_locale_string (fname) : SCM_BOOL_F;
- msg = scm_from_locale_string (lt_dlerror ());
- scm_misc_error (subr, "file: ~S, message: ~S", scm_list_2 (fn, msg));
- }
-
- return (void *) handle;
-}
-
-static void
-sysdep_dynl_unlink (void *handle, const char *subr)
-{
- if (lt_dlclose ((lt_dlhandle) handle))
- {
- scm_misc_error (subr, (char *) lt_dlerror (), SCM_EOL);
- }
-}
-
-static void *
-sysdep_dynl_value (const char *symb, void *handle, const char *subr)
+static SCM
+dlerror_string (const char *fallback)
{
- void *fptr;
-
- fptr = lt_dlsym ((lt_dlhandle) handle, symb);
- if (!fptr)
- scm_misc_error (subr, "Symbol not found: ~a",
- scm_list_1 (scm_from_locale_string (symb)));
- return fptr;
+ const char *message = dlerror ();
+ if (message)
+ return scm_from_locale_string (message);
+ return scm_from_utf8_string ("Unknown error");
}
-static void
-sysdep_dynl_init ()
-{
- char *env;
-
- lt_dlinit ();
-
- /* Initialize 'system_extensions_path' from
- $GUILE_SYSTEM_EXTENSIONS_PATH, or if that's not set:
- <SCM_LIB_DIR> <LT_PATHSEP_CHAR> <SCM_EXTENSIONS_DIR>.
-
- 'lt_dladdsearchdir' can't be used because it is searched before
- the system-dependent search path, which is the one 'libtool
- --mode=execute -dlopen' fiddles with (info "(libtool) Libltdl
- Interface"). See
- <http://lists.gnu.org/archive/html/guile-devel/2010-11/msg00095.html>.
-
- The environment variables $LTDL_LIBRARY_PATH and $LD_LIBRARY_PATH
- can't be used because they would be propagated to subprocesses
- which may cause problems for other programs. See
- <http://lists.gnu.org/archive/html/guile-devel/2012-09/msg00037.html> */
-
- env = getenv ("GUILE_SYSTEM_EXTENSIONS_PATH");
- if (env)
- system_extensions_path = env;
- else
- {
- system_extensions_path
- = scm_gc_malloc_pointerless (strlen (SCM_LIB_DIR)
- + strlen (SCM_EXTENSIONS_DIR) + 2,
- "system_extensions_path");
- sprintf (system_extensions_path, "%s%c%s",
- SCM_LIB_DIR, LT_PATHSEP_CHAR, SCM_EXTENSIONS_DIR);
- }
-}
-
-scm_t_bits scm_tc16_dynamic_obj;
-
-#define DYNL_FILENAME SCM_SMOB_OBJECT
-#define DYNL_HANDLE(x) ((void *) SCM_SMOB_DATA_2 (x))
-#define SET_DYNL_HANDLE(x, v) (SCM_SET_SMOB_DATA_2 ((x), (scm_t_bits) (v)))
-
-
-
-static int
-dynl_obj_print (SCM exp, SCM port, scm_print_state *pstate)
-{
- scm_puts ("#<dynamic-object ", port);
- scm_iprin1 (DYNL_FILENAME (exp), port, pstate);
- if (DYNL_HANDLE (exp) == NULL)
- scm_puts (" (unlinked)", port);
- scm_putc ('>', port);
- return 1;
-}
-
-
-SCM_DEFINE (scm_dynamic_link, "dynamic-link", 0, 1, 0,
- (SCM filename),
- "Find the shared object (shared library) denoted by\n"
- "@var{filename} and link it into the running Guile\n"
- "application. The returned\n"
- "scheme object is a ``handle'' for the library which can\n"
- "be passed to @code{dynamic-func}, @code{dynamic-call} etc.\n\n"
- "Searching for object files is system dependent. Normally,\n"
- "if @var{filename} does have an explicit directory it will\n"
- "be searched for in locations\n"
- "such as @file{/usr/lib} and @file{/usr/local/lib}.\n\n"
- "When @var{filename} is omitted, a @dfn{global symbol handle} is\n"
- "returned. This handle provides access to the symbols\n"
- "available to the program at run-time, including those exported\n"
- "by the program itself and the shared libraries already loaded.\n")
-#define FUNC_NAME s_scm_dynamic_link
+SCM_DEFINE_STATIC (scm_dlopen, "dlopen", 2, 0, 0, (SCM name, SCM flags), "")
+#define FUNC_NAME s_scm_dlopen
{
void *handle;
- char *file;
+ int c_flags = scm_to_int (flags);
- scm_dynwind_begin (0);
- scm_i_dynwind_pthread_mutex_lock (<dl_lock);
-
- if (SCM_UNBNDP (filename))
- file = NULL;
+ if (scm_is_false (name))
+ handle = dlopen (NULL, c_flags);
else
{
- file = scm_to_locale_string (filename);
- scm_dynwind_free (file);
+ char *c_name = scm_to_locale_string (name);
+ handle = dlopen (c_name, c_flags);
+ free (c_name);
}
- handle = sysdep_dynl_link (file, FUNC_NAME);
- scm_dynwind_end ();
+ if (!handle) {
+ SCM message = dlerror_string ("Unknown error while opening module");
+ SCM_MISC_ERROR ("file ~S, message ~S", scm_list_2 (name, message));
+ }
- SCM_RETURN_NEWSMOB2 (scm_tc16_dynamic_obj,
- SCM_UNBNDP (filename)
- ? SCM_UNPACK (SCM_BOOL_F) : SCM_UNPACK (filename),
- handle);
+ return scm_from_pointer (handle, NULL);
}
#undef FUNC_NAME
-
-SCM_DEFINE (scm_dynamic_object_p, "dynamic-object?", 1, 0, 0,
- (SCM obj),
- "Return @code{#t} if @var{obj} is a dynamic object handle,\n"
- "or @code{#f} otherwise.")
-#define FUNC_NAME s_scm_dynamic_object_p
+SCM_DEFINE_STATIC (scm_dlclose, "dlclose", 1, 0, 0, (SCM obj), "")
+#define FUNC_NAME s_scm_dlclose
{
- return scm_from_bool (SCM_TYP16_PREDICATE (scm_tc16_dynamic_obj, obj));
-}
-#undef FUNC_NAME
+ void *handle = scm_to_pointer (obj);
-
-SCM_DEFINE (scm_dynamic_unlink, "dynamic-unlink", 1, 0, 0,
- (SCM dobj),
- "Unlink a dynamic object from the application, if possible. The\n"
- "object must have been linked by @code{dynamic-link}, with \n"
- "@var{dobj} the corresponding handle. After this procedure\n"
- "is called, the handle can no longer be used to access the\n"
- "object.")
-#define FUNC_NAME s_scm_dynamic_unlink
-{
- /*fixme* GC-problem */
- SCM_VALIDATE_SMOB (SCM_ARG1, dobj, dynamic_obj);
-
- scm_dynwind_begin (0);
- scm_i_dynwind_pthread_mutex_lock (<dl_lock);
- if (DYNL_HANDLE (dobj) == NULL) {
- SCM_MISC_ERROR ("Already unlinked: ~S", scm_list_1 (dobj));
- } else {
- sysdep_dynl_unlink (DYNL_HANDLE (dobj), FUNC_NAME);
- SET_DYNL_HANDLE (dobj, NULL);
+ if (dlclose (handle) != 0) {
+ SCM message = dlerror_string ("Unknown error");
+ SCM_MISC_ERROR ("Error closing module: ~S", scm_list_1 (message));
}
- scm_dynwind_end ();
return SCM_UNSPECIFIED;
}
#undef FUNC_NAME
-
-SCM_DEFINE (scm_dynamic_pointer, "dynamic-pointer", 2, 0, 0,
- (SCM name, SCM dobj),
- "Return a ``wrapped pointer'' to the symbol @var{name}\n"
- "in the shared object referred to by @var{dobj}. The returned\n"
- "pointer points to a C object.\n\n"
- "Regardless whether your C compiler prepends an underscore\n"
- "@samp{_} to the global names in a program, you should\n"
- "@strong{not} include this underscore in @var{name}\n"
- "since it will be added automatically when necessary.")
-#define FUNC_NAME s_scm_dynamic_pointer
+SCM_DEFINE_STATIC (scm_dlsym, "dlsym", 2, 0, 0, (SCM obj, SCM name), "")
+#define FUNC_NAME s_scm_dlsym
{
- void *val;
+ void *handle = scm_to_pointer (obj);
+ char *c_name = scm_to_utf8_string (name);
- SCM_VALIDATE_STRING (1, name);
- SCM_VALIDATE_SMOB (SCM_ARG2, dobj, dynamic_obj);
+ void *sym = dlsym (handle, c_name);
+ free (c_name);
- if (DYNL_HANDLE (dobj) == NULL)
- SCM_MISC_ERROR ("Already unlinked: ~S", dobj);
- else
- {
- char *chars;
-
- scm_dynwind_begin (0);
- scm_i_dynwind_pthread_mutex_lock (<dl_lock);
- chars = scm_to_locale_string (name);
- scm_dynwind_free (chars);
- val = sysdep_dynl_value (chars, DYNL_HANDLE (dobj), FUNC_NAME);
- scm_dynwind_end ();
+ if (!sym) {
+ SCM message = dlerror_string ("Unknown error");
+ SCM_MISC_ERROR ("Error resolving ~S: ~S", scm_list_2 (name, message));
+ }
- return scm_from_pointer (val, NULL);
- }
+ return scm_from_pointer (sym, NULL);
}
#undef FUNC_NAME
+#define DEFINE_LAZY_VAR(c_name, mod_name, sym_name) \
+ static SCM c_name##_var; \
+ static void init_##c_name##_var (void) \
+ { \
+ c_name##_var = scm_c_public_lookup (mod_name, sym_name); \
+ } \
+ static SCM c_name (void) \
+ { \
+ static scm_i_pthread_once_t once = SCM_I_PTHREAD_ONCE_INIT; \
+ scm_i_pthread_once (&once, init_##c_name##_var); \
+ return scm_variable_ref (c_name##_var); \
+ }
-SCM_DEFINE (scm_dynamic_func, "dynamic-func", 2, 0, 0,
- (SCM name, SCM dobj),
- "Return a ``handle'' for the function @var{name} in the\n"
- "shared object referred to by @var{dobj}. The handle\n"
- "can be passed to @code{dynamic-call} to actually\n"
- "call the function.\n\n"
- "Regardless whether your C compiler prepends an underscore\n"
- "@samp{_} to the global names in a program, you should\n"
- "@strong{not} include this underscore in @var{name}\n"
- "since it will be added automatically when necessary.")
-#define FUNC_NAME s_scm_dynamic_func
+DEFINE_LAZY_VAR (load_foreign_library,
+ "system foreign-library", "load-foreign-library");
+DEFINE_LAZY_VAR (foreign_library_p,
+ "system foreign-library", "foreign-library?");
+DEFINE_LAZY_VAR (foreign_library_pointer,
+ "system foreign-library", "foreign-library-pointer");
+
+SCM
+scm_dynamic_link (SCM filename)
{
- return scm_dynamic_pointer (name, dobj);
+ return scm_call_1 (load_foreign_library (), filename);
}
-#undef FUNC_NAME
+SCM
+scm_dynamic_object_p (SCM obj)
+{
+ return scm_call_1 (foreign_library_p (), obj);
+}
-SCM_DEFINE (scm_dynamic_call, "dynamic-call", 2, 0, 0,
- (SCM func, SCM dobj),
- "Call a C function in a dynamic object. Two styles of\n"
- "invocation are supported:\n\n"
- "@itemize @bullet\n"
- "@item @var{func} can be a function handle returned by\n"
- "@code{dynamic-func}. In this case @var{dobj} is\n"
- "ignored\n"
- "@item @var{func} can be a string with the name of the\n"
- "function to call, with @var{dobj} the handle of the\n"
- "dynamic object in which to find the function.\n"
- "This is equivalent to\n"
- "@smallexample\n\n"
- "(dynamic-call (dynamic-func @var{func} @var{dobj}) #f)\n"
- "@end smallexample\n"
- "@end itemize\n\n"
- "In either case, the function is passed no arguments\n"
- "and its return value is ignored.")
-#define FUNC_NAME s_scm_dynamic_call
+SCM
+scm_dynamic_pointer (SCM name, SCM obj)
{
- void (*fptr) (void);
+ return scm_call_2 (foreign_library_pointer (), obj, name);
+}
- if (scm_is_string (func))
- func = scm_dynamic_func (func, dobj);
- SCM_VALIDATE_POINTER (SCM_ARG1, func);
+SCM
+scm_dynamic_func (SCM name, SCM obj)
+{
+ return scm_dynamic_pointer (name, obj);
+}
- fptr = SCM_POINTER_VALUE (func);
- fptr ();
+SCM
+scm_dynamic_call (SCM name, SCM obj)
+{
+ SCM pointer = scm_dynamic_pointer (name, obj);
+ void (*f)(void) = SCM_POINTER_VALUE (pointer);
+ f();
return SCM_UNSPECIFIED;
}
-#undef FUNC_NAME
-void
-scm_init_dynamic_linking ()
+static void
+scm_init_system_foreign_library (void *unused)
{
- scm_tc16_dynamic_obj = scm_make_smob_type ("dynamic-object", 0);
- scm_set_smob_print (scm_tc16_dynamic_obj, dynl_obj_print);
+ scm_c_define ("RTLD_LAZY", scm_from_int (RTLD_LAZY));
+ scm_c_define ("RTLD_NOW", scm_from_int (RTLD_NOW));
+ scm_c_define ("RTLD_GLOBAL", scm_from_int (RTLD_GLOBAL));
+ scm_c_define ("RTLD_LOCAL", scm_from_int (RTLD_LOCAL));
- /* Make LTDL_LOCK recursive so that a pre-unwind handler can still use
- 'dynamic-link', as is the case at the REPL. See
- <https://bugs.gnu.org/29275>. */
- scm_i_pthread_mutex_init (<dl_lock,
- scm_i_pthread_mutexattr_recursive);
-
- sysdep_dynl_init ();
#include "dynl.x"
}
+
+void
+scm_init_dynamic_linking ()
+{
+ scm_c_register_extension ("libguile-" SCM_EFFECTIVE_VERSION,
+ "scm_init_system_foreign_library",
+ scm_init_system_foreign_library,
+ NULL);
+
+ // FIXME: Deprecate all of these, once (system foreign-library) has
+ // had enough time in the world.
+ scm_c_define_gsubr
+ ("dynamic-link", 0, 1, 0, (scm_t_subr) scm_dynamic_link);
+ scm_c_define_gsubr
+ ("dynamic-object?", 1, 0, 0, (scm_t_subr) scm_dynamic_object_p);
+ scm_c_define_gsubr
+ ("dynamic-func", 2, 0, 0, (scm_t_subr) scm_dynamic_func);
+ scm_c_define_gsubr
+ ("dynamic-pointer", 2, 0, 0, (scm_t_subr) scm_dynamic_pointer);
+ scm_c_define_gsubr
+ ("dynamic-call", 2, 0, 0, (scm_t_subr) scm_dynamic_call);
+}
diff --git a/libguile/dynl.h b/libguile/dynl.h
index 3178c9a..dd10bf4 100644
--- a/libguile/dynl.h
+++ b/libguile/dynl.h
@@ -1,7 +1,7 @@
#ifndef SCM_DYNL_H
#define SCM_DYNL_H
-/* Copyright 1996,1998,2000-2001,2006,2008,2010,2018
+/* Copyright 1996,1998,2000-2001,2006,2008,2010,2018,2021
Free Software Foundation, Inc.
This file is part of Guile.
@@ -27,11 +27,10 @@
SCM_API SCM scm_dynamic_link (SCM fname);
-SCM_API SCM scm_dynamic_unlink (SCM dobj);
SCM_API SCM scm_dynamic_object_p (SCM obj);
-SCM_API SCM scm_dynamic_pointer (SCM name, SCM dobj);
-SCM_API SCM scm_dynamic_func (SCM symb, SCM dobj);
-SCM_API SCM scm_dynamic_call (SCM symb, SCM dobj);
+SCM_API SCM scm_dynamic_pointer (SCM name, SCM obj);
+SCM_API SCM scm_dynamic_func (SCM name, SCM obj);
+SCM_API SCM scm_dynamic_call (SCM name, SCM obj);
SCM_INTERNAL void scm_init_dynamic_linking (void);
diff --git a/libguile/extensions.c b/libguile/extensions.c
index a094159..61c975e 100644
--- a/libguile/extensions.c
+++ b/libguile/extensions.c
@@ -1,4 +1,4 @@
-/* Copyright 2001,2002,2004,2006,2009-2011,2018-2019
+/* Copyright 2001,2002,2004,2006,2009-2011,2018-2019,2021
Free Software Foundation, Inc.
This file is part of Guile.
@@ -27,6 +27,7 @@
#include "dynwind.h"
#include "gc.h"
#include "gsubr.h"
+#include "foreign.h"
#include "strings.h"
#include "threads.h"
@@ -113,7 +114,9 @@ load_extension (SCM lib, SCM init)
/* Dynamically link the library. */
#if HAVE_MODULES
- scm_dynamic_call (init, scm_dynamic_link (lib));
+ SCM pointer = scm_dynamic_pointer (init, scm_dynamic_link (lib));
+ void (*f)(void) = scm_to_pointer (pointer);
+ f ();
#else
scm_misc_error ("load-extension",
"extension ~S:~S not registered and dynamic-link disabled",
diff --git a/libguile/guile.c b/libguile/guile.c
index ae592ed..bafe5d6 100644
--- a/libguile/guile.c
+++ b/libguile/guile.c
@@ -1,4 +1,4 @@
-/* Copyright 1996-1997,2000-2001,2006,2008,2011,2013,2018
+/* Copyright 1996-1997,2000-2001,2006,2008,2011,2013,2018,2021
Free Software Foundation, Inc.
This file is part of Guile.
@@ -28,7 +28,6 @@
# include <config.h>
#endif
-#include <ltdl.h>
#include <locale.h>
#include <stdio.h>
diff --git a/module/Makefile.am b/module/Makefile.am
index 45113b5..86d5401 100644
--- a/module/Makefile.am
+++ b/module/Makefile.am
@@ -1,6 +1,6 @@
## Process this file with automake to produce Makefile.in.
##
-## Copyright (C) 2009-2020 Free Software Foundation, Inc.
+## Copyright (C) 2009-2021 Free Software Foundation, Inc.
##
## This file is part of GUILE.
##
@@ -334,7 +334,7 @@ SOURCES = \
system/base/ck.scm \
\
system/foreign.scm \
- \
+ system/foreign-library.scm \
system/foreign-object.scm \
\
system/repl/debug.scm \
diff --git a/module/oop/goops.scm b/module/oop/goops.scm
index df6df4f..9edc16b 100644
--- a/module/oop/goops.scm
+++ b/module/oop/goops.scm
@@ -1,6 +1,6 @@
;;;; goops.scm -- The Guile Object-Oriented Programming System
;;;;
-;;;; Copyright (C) 1998-2003,2006,2009-2011,2013-2015,2017-2018
+;;;; Copyright (C) 1998-2003,2006,2009-2011,2013-2015,2017-2018,2021
;;;; Free Software Foundation, Inc.
;;;; Copyright (C) 1993-1998 Erick Gallesio - I3S-CNRS/ESSI <eg@unice.fr>
;;;;
@@ -3307,10 +3307,15 @@ var{initargs}."
(define <directory> (find-subclass <top> '<directory>))
(define <array> (find-subclass <top> '<array>))
(define <character-set> (find-subclass <top> '<character-set>))
-(define <dynamic-object> (find-subclass <top> '<dynamic-object>))
(define <guardian> (find-subclass <applicable> '<guardian>))
(define <macro> (find-subclass <top> '<macro>))
+;; <dynamic-object> used to be a SMOB type, albeit not exported even to
+;; C. However now it's a record type, though still private. Cross our
+;; fingers that nobody is using it in anger!
+(define <dynamic-object>
+ (module-ref (resolve-module '(system foreign-library)) '<foreign-library>))
+
(define (define-class-subtree class)
(define! (class-name class) class)
(for-each define-class-subtree (class-direct-subclasses class)))
diff --git a/module/system/foreign-library.scm
b/module/system/foreign-library.scm
new file mode 100644
index 0000000..6945fca
--- /dev/null
+++ b/module/system/foreign-library.scm
@@ -0,0 +1,231 @@
+;;; Dynamically linking foreign libraries via dlopen and dlsym
+;;; Copyright (C) 2021 Free Software Foundation, Inc.
+;;;
+;;; This library is free software: you can redistribute it and/or modify
+;;; it under the terms of the GNU Lesser General Public License as
+;;; published by the Free Software Foundation, either version 3 of the
+;;; License, or (at your option) any later version.
+;;;
+;;; This library is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+;;; Lesser General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU Lesser General Public
+;;; License along with this program. If not, see
+;;; <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+;;;
+;;; Implementation of dynamic-link.
+;;;
+;;; Code:
+
+
+(define-module (system foreign-library)
+ #:use-module (ice-9 match)
+ #:use-module (srfi srfi-9)
+ #:use-module (system foreign)
+ #:export (guile-extensions-path
+ ltdl-library-path
+ guile-system-extensions-path
+
+ load-foreign-library
+ foreign-library?
+ foreign-library-pointer
+ foreign-library-function))
+
+(define-record-type <foreign-library>
+ (make-foreign-library filename handle)
+ foreign-library?
+ (filename foreign-library-filename)
+ (handle foreign-library-handle set-foreign-library-handle!))
+
+(eval-when (expand load eval)
+ (load-extension (string-append "libguile-" (effective-version))
+ "scm_init_system_foreign_library"))
+
+(define system-library-extensions
+ (cond
+ ((string-contains %host-type "-darwin-")
+ '(".bundle" ".so" ".dylib"))
+ ((or (string-contains %host-type "cygwin")
+ (string-contains %host-type "mingw")
+ (string-contains %host-type "msys"))
+ '(".dll"))
+ (else
+ '(".so"))))
+
+(define (has-extension? head exts)
+ (match exts
+ (() #f)
+ ((ext . exts)
+ (or (string-contains head ext)
+ (has-extension? head exts)))))
+
+(define (file-exists-with-extension head exts)
+ (if (has-extension? head exts)
+ (and (file-exists? head) head)
+ (let lp ((exts exts))
+ (match exts
+ (() #f)
+ ((ext . exts)
+ (let ((head (string-append head ext)))
+ (if (file-exists? head)
+ head
+ (lp exts))))))))
+
+(define (file-exists-in-path-with-extension basename path exts)
+ (match path
+ (() #f)
+ ((dir . path)
+ (or (file-exists-with-extension (in-vicinity dir basename) exts)
+ (file-exists-in-path-with-extension basename path exts)))))
+
+(define path-separator
+ (case (system-file-name-convention)
+ ((posix) #\:)
+ ((windows) #\;)
+ (else (error "unreachable"))))
+
+(define (parse-path var)
+ (match (getenv var)
+ (#f #f)
+ ;; Ignore e.g. "export GUILE_SYSTEM_EXTENSIONS_PATH=".
+ ("" '())
+ (val (string-split val path-separator))))
+
+(define guile-extensions-path
+ (make-parameter
+ (or (parse-path "GUILE_EXTENSIONS_PATH") '())))
+
+(define ltdl-library-path
+ (make-parameter
+ (or (parse-path "LTDL_LIBRARY_PATH") '())))
+
+(define guile-system-extensions-path
+ (make-parameter
+ (or (parse-path "GUILE_SYSTEM_EXTENSIONS_PATH")
+ (list (assq-ref %guile-build-info 'libdir)
+ (assq-ref %guile-build-info 'extensionsdir)))))
+
+;; There are a few messy situations here related to libtool.
+;;
+;; Guile used to use libltdl, the dynamic library loader provided by
+;; libtool. This loader used LTDL_LIBRARY_PATH, and for backwards
+;; compatibility we still support that path.
+;;
+;; However, libltdl would not only open ".so" (or ".dll", etc) files,
+;; but also the ".la" files created by libtool. In installed libraries
+;; -- libraries that are in the target directories of "make install" --
+;; .la files are never needed, to the extent that most GNU/Linux
+;; distributions remove them entirely. It is sufficient to just load
+;; the ".so" (or ".dll", etc) files.
+;;
+;; But for uninstalled dynamic libraries, like those in a build tree, it
+;; is a bit of a mess. If you have a project that uses libtool to build
+;; libraries -- which is the case for Guile, and for most projects using
+;; autotools -- and you build foo.so in directory D, libtool will put
+;; foo.la in D, but foo.so goes in D/.libs.
+;;
+;; The nice thing about ltdl was that it could load the .la file, even
+;; from a build tree, preventing the existence of ".libs" from leaking
+;; out to the user.
+;;
+;; We don't use libltdl now, essentially for flexibility and
+;; error-reporting reasons. But, it would be nice to keep this old
+;; use-case working. So as a stopgap solution, we add a ".libs" subdir
+;; to the path for each entry in LTDL_LIBRARY_PATH, in case the .so is
+;; there instead of alongside the .la file.
+(define (augment-ltdl-library-path path)
+ (match path
+ (() '())
+ ((dir . path)
+ (cons* dir (in-vicinity dir ".libs")
+ (augment-ltdl-library-path path)))))
+
+(define (default-search-path search-ltdl-library-path?)
+ (append
+ (guile-extensions-path)
+ (if search-ltdl-library-path?
+ (augment-ltdl-library-path (ltdl-library-path))
+ '())
+ (guile-system-extensions-path)))
+
+(define* (load-foreign-library #:optional filename #:key
+ (extensions system-library-extensions)
+ (search-ltdl-library-path? #t)
+ (search-path (default-search-path
+ search-ltdl-library-path?))
+ (search-system-paths? #t)
+ (lazy? #t) (global? #f))
+ (define (error-not-found)
+ (scm-error 'misc-error "load-foreign-library"
+ "file: ~S, message: ~S"
+ (list filename "file not found")
+ #f))
+ (define flags
+ (logior (if lazy? RTLD_LAZY RTLD_NOW)
+ (if global? RTLD_GLOBAL RTLD_LOCAL)))
+ (define (dlopen* name) (dlopen name flags))
+ (make-foreign-library
+ filename
+ (cond
+ ((not filename)
+ ;; The self-open trick.
+ (dlopen* #f))
+ ((or (absolute-file-name? filename)
+ (string-any file-name-separator? filename))
+ (cond
+ ((or (file-exists-with-extension filename extensions)
+ (and search-ltdl-library-path?
+ (file-exists-with-extension
+ (in-vicinity (in-vicinity (dirname filename) ".libs")
+ (basename filename))
+ extensions)))
+ => dlopen*)
+ (else
+ (error-not-found))))
+ ((file-exists-in-path-with-extension filename search-path extensions)
+ => dlopen*)
+ (search-system-paths?
+ (if (or (null? extensions) (has-extension? filename extensions))
+ (dlopen* filename)
+ (let lp ((extensions extensions))
+ (match extensions
+ ((extension)
+ ;; Open in tail position to propagate any exception.
+ (dlopen* (string-append filename extension)))
+ ((extension . extensions)
+ ;; If there is more than one extension, unfortunately we
+ ;; only report the error for the last extension. This is
+ ;; not great because maybe the library was found with the
+ ;; first extension, failed to load and had an interesting
+ ;; error, but then we swallowed that interesting error and
+ ;; proceeded, eventually throwing a "file not found"
+ ;; exception. FIXME to use more structured exceptions and
+ ;; stop if the error that we get is more specific than
+ ;; just "file not found".
+ (or (false-if-exception
+ (dlopen* (string-append filename extension)))
+ (lp extensions)))))))
+ (else
+ (error-not-found)))))
+
+(define (->foreign-library lib)
+ (if (foreign-library? lib)
+ lib
+ (load-foreign-library lib)))
+
+(define* (foreign-library-pointer lib name)
+ (let ((handle (foreign-library-handle (->foreign-library lib))))
+ (dlsym handle name)))
+
+(define* (foreign-library-function lib name
+ #:key
+ (return-type void)
+ (arg-types '())
+ (return-errno? #f))
+ (let ((pointer (foreign-library-pointer lib name)))
+ (pointer->procedure return-type pointer arg-types
+ #:return-errno? return-errno?)))
diff --git a/test-suite/tests/foreign.test b/test-suite/tests/foreign.test
index 67b5c37..966d214 100644
--- a/test-suite/tests/foreign.test
+++ b/test-suite/tests/foreign.test
@@ -1,6 +1,6 @@
;;;; foreign.test --- FFI. -*- mode: scheme; coding: utf-8; -*-
;;;;
-;;;; Copyright (C) 2010, 2011, 2012, 2013, 2017 Free Software Foundation,
Inc.
+;;;; Copyright (C) 2010, 2011, 2012, 2013, 2017, 2021 Free Software
Foundation, Inc.
;;;;
;;;; This library is free software; you can redistribute it and/or
;;;; modify it under the terms of the GNU Lesser General Public
@@ -21,6 +21,7 @@
;;;
(define-module (test-foreign)
+ #:use-module (system foreign-library)
#:use-module (system foreign)
#:use-module (rnrs bytevectors)
#:use-module (srfi srfi-1)
@@ -29,12 +30,13 @@
#:use-module (test-suite lib))
-(with-test-prefix "dynamic-pointer"
+(with-test-prefix "foreign-library-pointer"
(pass-if-exception
"error message"
- '(misc-error . "^Symbol not found")
- (dynamic-func "does_not_exist___" (dynamic-link))))
+ ;; The error comes from dlsym, which is system-dependent.
+ '(misc-error . "")
+ (foreign-library-pointer #f "does_not_exist___")))
(with-test-prefix "null pointer"
@@ -73,7 +75,7 @@
(pass-if "equal? modulo finalizer"
(let ((finalizer (false-if-exception
- (dynamic-func "scm_is_pair" (dynamic-link)))))
+ (foreign-library-pointer #f "scm_is_pair"))))
(if (not finalizer)
(throw 'unresolved) ; Windows or a static build
(equal? (make-pointer 123)
@@ -81,7 +83,7 @@
(pass-if "equal? modulo finalizer (set-pointer-finalizer!)"
(let ((finalizer (false-if-exception
- (dynamic-func "scm_is_pair" (dynamic-link))))
+ (foreign-library-pointer #f "scm_is_pair")))
(ptr (make-pointer 123)))
(if (not finalizer)
(throw 'unresolved) ; Windows or a static build
@@ -232,19 +234,15 @@
;; linking with `-export-dynamic'. Just skip these tests when it's
;; not visible.
(false-if-exception
- (pointer->procedure void
- (dynamic-func "qsort"
- (cond
- ((string-contains %host-type "cygwin")
- ;; On Cygwin, dynamic-link does
- ;; not search recursively into
- ;; linked DLLs. Thus, one needs
- ;; to link to the core C
- ;; library DLL explicitly.
- (dynamic-link "cygwin1"))
- (else
- (dynamic-link))))
- (list '* size_t size_t '*))))
+ (foreign-library-function
+ (cond
+ ((string-contains %host-type "cygwin")
+ ;; On Cygwin, load-foreign-library does not search recursively
+ ;; into linked DLLs. Thus, one needs to link to the core C
+ ;; library DLL explicitly.
+ "cygwin1")
+ (else #f))
+ "qsort" #:arg-types (list '* size_t size_t '*))))
(define (dereference-pointer-to-byte ptr)
(let ((b (pointer->bytevector ptr 1)))