[Chicken-users] The Documentation Problem

[Matt Gushee]

Hello, folks--

I have been thinking about what I will call The Documentation Problem. 
I'm sure this is not news to anyone, but just to clarify exactly what I 
mean: it seems to me that Chicken Scheme, as well as Scheme in general, 
is much harder to learn than it should be, due in large part to the 
fragmentary nature of the documentation.

And I conceived of a project that might make it easier to access all the 
various docs. I call it SUDS (Scheme Unified Documentation Server). 
Basically, there would be a lightweight Web server (maybe Spiffy-based?) 
that could be deployed either on the public internet or on your own 
computer. It would have a database of all the reference docs for a 
particular system ... so for Chicken, the DB would include R5RS, all 
supported SRFIs, the Chicken manual, and docs for all the eggs. 
Furthermore, everything would be stored in per-symbol chunks, so you 
could, for example, view a web-based index of all available symbols, 
with each name linked to the detailed doc for that symbol. Or you could 
have a Vim plugin that would request info on the function nearest the 
cursor, etc.

However ... I think it is crucial to be able to auto-update the 
database. I thought (naively, as it now seems) that the SUDS server 
could accomplish that by grabbing the latest online HTML version of 
whatever doc and parsing that in an appropriate fashion ... uh, well, 
not so easy. Maybe not feasible. Because after examining a few docs and 
a couple of parsing experiments, I have realized that

 * different doc collections use different structural conventions;

 * with few exceptions, there is no semantics in the HTML (i.e. nothing
   like <div class="function-description" id="cons"> that would
   facilitate automated data extraction); and

 * some of the pages are invalid HTML (for example, I tried parsing an
   R5RS page with the html-parser egg, and got strange results--so then
   I tried validating it w/ the W3C online validator--yeow! Let's just
   say it was rather scary to see how many errors came up).

So, at this point I don't see a way to make SUDS work with a reasonable 
effort. But I'd like to throw out some questions to the community:

- Do you think we should have something like SUDS?

- Would *you* use it?

- Any ideas as to how it could actually be done?

--
Matt Gushee
: Bantam - lightweight file manager : matt.gushee.net/software/bantam/ :
: RASCL's A Simple Configuration Language :     matt.gushee.net/rascl/ :