Re: [Bug-apl] A call for justification of feature / library / extension proposals (was Re: Tk/Tcl interface)

[David B. Lamkins]

Thank you Elias and Jürgen for your comments regarding namespaces.

I'll say at the outset that, having had time to think about my proposal
overnight, I agree that that adding namespace support to the interpreter
is not as good an idea as I had first envisioned. It turns out that such
a "simple" change really does impact a huge swath of code. That in
itself is not a reason to avoid the change. However, it's likely that
the design and implementation would need to go through many iterations
in order to resolve all of the unanticipated effects. Let someone else
(NARS? Dyalog?) pave that road...

Jürgen, I personally wouldn't argue for against any language feature
based upon its implementation in C++ (except perhaps to say "don't do
anything that C++ has done"). ;)

Before letting this go entirely, though, I'd like to clarify one
important distinction. Namespaces in C++ are (unless something has
changed in many years since I last looked) a static construction. Please
correct me if I have this wrong, but one of the problems solved by C++
namespaces is that of attempting to avoid collisions between precompiled
libraries. (Jürgen's comment [1a] suggests to me that this is still the
case.)

The situation with dynamic lanuages (e.g. Common Lisp and Dyalog APL) is
materially different in that the namespace is (in the case of Dyalog
APL) or may be (in the case of Common Lisp) bound by the application
programmer rather than by the library developer, thus avoiding the
problem of how to resolve the case that multiple library developers have
independently chosen the same "obvious" prefix for their exported
symbols.

There are other ways to address the issue, of course. Java has the
particularly heinous (IMO) approach of using the notion of
"ownership" (via a verbose path of dot-separated delimiters) to create
an illusion of namespace separation. This is all well and good (if
you're willing to overlook the verbosity) until such time as the new
maintainer of a package decides to change identifying path. At that
point, all users of the package have to migrate their systems to the new
version. Java developers seem to thrive on keeping up with changes in
their platform; I prefer stability.

As far as native functions in APL: Yes, their namespace is a non-issue.
However, I see it as likely that nearly every library of native
functions will come with an APL wrapper. At a bare minimum, the wrapper
is likely to provide syntactic sugaring. (I quickly threw together my
own wrapper to experiment with the file_io library; I'm sure others have
done something similar.) At any rate, its the associated wrapper that
one needs to be concerned about.

Of course,  given APL's flat namespace, there's still the possibility of
collisions between identifiers in independently written code. I'm not
fully convinced by Elias' argument that Emacs has managed for decades
using only a prefix naming convention. I believe that the use cases
(between APL programming and Elisp extensions) are sufficiently
different that we ought to be preemptively concerned about collisions
among APL packages.

Emacs has a curated collection of extensions that may serve as a handy
reference for future Elisp coders. A quick grep over /usr/share/emacs is
a good first test as to whether your proposed package name is likely to
collide with some other programmer's work. (Yes, there are packages not
distributed with Emacs yet... Most of these can be found in the Emacs
WiKi - another centralized repository.)

A centralized repository of GNU APL programs would benefit the community
and serve much the same function as the Elisp source distribution and
the Emacs WiKi. Perhaps someone will take on the challenges of setting
up such a centralized repository...

Another thing we could do is to suggest some conventions to be used by
APL authors along with some tools that understand those conventions. For
example, if we adopt a prefix naming approach (perhaps a short
alphabetic sequence terminated by an APL del or del-underbar), then we
can also provide a tool (coded in APL, probably) to rewrite the prefixes
in a package in the event that it becomes necessary to avoid a name
collision. Just a thought...

Elias, I like your thinking behind standardizing extensions. I think
that would be a fruitful concept to pursue further. We really are
dealing with a multi-language (APL, compiled libraries, elisp,
Javascript, ...), multi-platform (Linux, OS X, Solaris, Windows),
multi-process (APs, servers, system services, ...) system. I think we
should attempt to engineer a packaging solution that is, to the greatest
extent possible, portable in IBM/ISO APL rather than leaning toward any
one existing packaging methodology. I'm not entirely certain how this'd
work, but I have some preliminary ideas that I think would fit your
vision...

Before I get back to work, I'd like to leave you all with a thought
regarding the size of the APL community. Back in the mid `90s I wrote a
tutorial book on Common Lisp programming ("Successful Lisp: How To
Understand and Use Common Lisp"), mainly to cement my own understanding
after having used the language for experimental programming and
prototyping for about ten years. At the time, there wasn't a big Common
Lisp community, but rather a small collection of old-timers (the folks
from Stanford, MIT and a few commercial spin-offs) and younger
enthusiasts (me among them). When I was approached by Markus Fix to
publish a dead-tree edition of the book, I cautioned that there were
"maybe a hundred" potential customers. 

"Successful Lisp" came out in late 2004 and sold several hundred copies
in its first year. The book has continued to sell 150 to 200 copies
every year since then. Coincidentally (and I'm certain: not on the
strength of my book), we've seen the Common Lisp community grow by leaps
and bounds. 

Better tools, I believe, were a driving factor in the growth of the
Common Lisp community. CMUCL, which was quite difficult to compile (let
alone port) was the leading open-source Common Lisp in 2004. CMUCL has
since been supplanted by SBCL which is available on more platforms and
quite simple to compile from source. 

I suspect that other books (Norvig's for example, which provided
concrete examples of Common Lisp applications) helped to bring a lot of
newcomers into the Common Lisp community.

Later, Quicklisp provided the community with a centralized collection of
libraries. The important contribution of Quicklisp, I believe, is that
it gathered many far-flung libraries, verified that they at least
compiled, and made them all available from a central repository along
with some primitive metadata and dependency management. Quicklisp isn't
a *great* package manager, but it beats the heck out of iterative Google
searches.

I believe that GNU APL places the APL community on the same cusp that
the Common Lisp community found itself in the `00s. There's now a
complete implementation of the language that's easy to install and use.
This will eliminate the financial barrier to entry for people who might
otherwise have to choose between getting involved in the community or
spending $50 (or more; possibly much more depending upon intended use)
for a commercial APL system. The use of Unicode for APL finally resolves
the technical challenges of typing and reading APL code on a commodity
computer.

I don't think that there's a large existing body of freely-available APL
code as there was with Common Lisp. It'd be nice, though, if we could
get some kind of package management system (and/or conventions) in place
earlier rather than later.