Re: [Chicken-users] Understanding modules?

[Norman Gray]

Greetings.

On 16 Mar 2016, at 17:01, Norman Gray wrote:

> Thank you Oleg and John, for these explanations.  I won't be back in 
> Chicken-land for a couple of days, but as soon as I can I'll aim to 
> understand this point well enough to offer a draft of additional text 
> for the manual.

A little while ago, I was asking some new-user questions about modules, 
and received very useful clarifications from John Cowan and Oleg Kolosov 
-- for which, many thanks.  Finally, I have a chance to rework these 
into some suggested changes for the corresponding parts of the Chicken 
manual.  The text below includes some additional text, and mentions some 
remaining puzzles.

The page <http://wiki.call-cc.org/man/4/Basic%20mode%20of%20operation> 
says that 'The most portable way of creating separately linkable 
entities is supported by so-called units', and describes the (declare 
(uses UNITNAME)) expression.  The next page, 
<http://wiki.call-cc.org/man/4/Using%20the%20compiler>, includes a 
section 'An example with multiple files', which illustrates a process 
for combining multiple bodies of code into a single executable, using 
units -- (declare (uses bar)).  This gives the strong impression that 
using units is the preferred, or idiomatic, way of combining separate 
bodies of code.

However John remarked, regarding units, that 'IMAO, you don't need to 
know anything about them unless you are writing code to be incorporated 
into the main Chicken build, which is unlikely.'  In the same thread 
Oleg said that the use of unit files 'is generally frowned upon.'  In 
that case, is it desirable to have them appear so early and so 
prominently in the Chicken manual?  I appreciate it may be the logical 
place from an implementation point of view.

I get the impression from the list that it is instead modules that are 
the preferred way of combining bodies of code; if so, then it might be 
useful to remove the discussion of units in the 'Using the compiler' 
page, or at least note that this is a low-level feature which should be 
used in practice only via modules, and pointing to 
<http://wiki.call-cc.org/man/4/Modules>.

The 'Modules' page could I think be a little clearer about the 
distinctions between importing, loading and using modules.  If these 
distinctions aren't clear in the reader's head (well, _this_ reader's 
head, at least), then the rest of the text is rather harder work than it 
need be.  Can I suggest something like the following text, which is 
heavily based on John's and Oleg's replies in this thread?  This would 
be located just before the documentation of the 'module' form in this 
page.

vvvv

A module may _export_ a subset of the bindings defined in its body, 
making those bindings visible to other code which _imports_ the module, 
at compilation time.  A module must be _loaded_ into a CHICKEN 
interpreter before its bindings can be imported; this loading can happen 
explicitly, or more commonly as a side-effect of some higher level 
expression, as below; this loading happens at run-time.  In CHICKEN, the 
loadable object can be either the Scheme source code in a `.scm` file, 
or a shared object compiled with `csc -shared`.

If, as is usually the case, you compile code which depends on other 
modules, then you must use an _import library_; this is generated by 
`csc` when given the option `-emit-import-library`.  In the most common 
case where a file foo.scm contains a single (module foo ...) form, the 
compilation should be done with `-emit-all-import-libraries`, which 
generates a file `foo.import.scm` named after the module.  In that case, 
the expression `(use foo)` (or equivalently `(require-extension foo)`) 
will automatially handle finding and loading the library, and importing 
its symbols.

The `(module ...)` form is documented below.  See also the discussion of 
`require-extension` and `use` on 
<http://wiki.call-cc.org/man/4/Non-standard%20macros%20and%20special%20forms>, 
and possibly the discussion of `require` on 
<http://wiki.call-cc.org/man/4/Unit%20eval#loading-extension-libraries>

^^^^

The 'Modules' page suggests that we 'follow the general rule of (import 
chicken scheme) (use anything-else)'.  But this is the only mention of 
`(use ...)` on this page.  Perhaps the link above would resolve this.

Can I suggest avoiding the module name 'test' in 'Examples of using 
modules' -- there is already an egg called 'test', and thus a file 
test.import.so in the egg tree, so there is a certain amount of scope 
for path confusion (ahem).

It would be reassuring, at the end of the subsection 'Examples of using 
modules', to include an example of using the `hello.scm` module in 
compiled code.  With `hello.scm` changed so that the module is named 
`hello` rather than `test`:

    % csc -emit-all-import-libraries -shared hello.scm
    % ls hello*
    hello.import.scm    hello.scm    hello.so
    % cat main.scm
    (use hello)
    (hello)
    % csc -o main main.scm
    % ./main
    Hello, world !
    %

Though I can load and import a module called `test` in a file 
`hello.scm` into csi, I wasn't able to work out how to compile and link 
this module using csc.  Possibly that's not important to be able to do, 
as long as one sticks to declaring module `foo` only in file `foo.scm`.

A puzzle: in his message John mentioned that 'Use [...] installs the 
file and imports it into the current module.'  I'm puzzled at this use 
of 'install'.  This appears to be talking of installation as a variant 
of loading (perhaps loading specifically an import library?).  However 
the only relevant mention of 'install' that I can see in the manual is 
in <http://wiki.call-cc.org/man/4/Extensions>, where it refers to 
libraries being installed in a certain place in the filesystem by 
`chicken-install`.  Am I just over-thinking this?

Deployment: Either in the 'Modules' page (along with the other examples 
at the end) or in the 'Deployment' page, it would be useful to show how 
to deploy a program using modules.  The obvious things don't appear to 
work:

    % head -3 hello.scm
    (module hello (hello greet)
      (import scheme)
      (define-syntax greet
    % csc -emit-all-import-libraries -shared hello.scm
    % cat main.scm
    (use hello)
    (hello)
    % csc -deploy main.scm
    % main/main -:d
    [debug] application startup...
    [debug] heap resized to 1048576 bytes
    [debug] stack bottom is 0x7fff59d8f780.
    [debug] entering toplevel toplevel...
    [debug] entering toplevel library_toplevel...
    [debug] entering toplevel build_2dversion_toplevel...
    [debug] entering toplevel eval_toplevel...
    [debug] entering toplevel expand_toplevel...
    [debug] entering toplevel modules_toplevel...
    [debug] resizing mutation-stack from 8k to 16k ...
    [debug] entering toplevel chicken_2dsyntax_toplevel...
    ; loading ./hello.so ...
    [debug] loading compiled module `./hello.so' (handle is 
0x00007f9b59d0ddb0)
    [panic] nursery is too small - try higher setting using the `-:s' 
option - execution terminated
    % otool -L main/main
    main/main:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 
(compatibility version 150.0.0, current version 1256.14.0)
	@executable_path/libchicken.dylib (compatibility version 1.0.0, current 
version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 1226.10.1)
    %

Trying, for example, `main/main -:s10000000` has the same effect; as 
does picking large numbers for `csc -nursery 1000000` when compiling.  
Googling 'chicken nursery size' gives me more information about animal 
husbandry than I can really use.

(I'm on OS X, 10.10).

I've assembled a short list of minor buglets in eggs and egg 
documentation.  Is it best if I report them here, or should I ask for an 
account on http://bugs.call-cc.org ?

Best wishes,

Norman


--
Norman Gray  :  https://nxg.me.uk
SUPA School of Physics and Astronomy, University of Glasgow, UK