Re: proposal for doc+web sources

[John Mandereau]

Le samedi 11 juillet 2009 à 03:21 -0700, Graham Percival a écrit :
> Here's my proposal for the source/makefile view of documentation.
> (this is the big argument one)

...and here is my big reply :-P


> My understanding is that linking between texinfo manuals is easier
> if the main files are in the same directory.

This is true. This also makes makefiles maintenance easier too, and it
most probably speeds up the doc build on multicore/multi-CPU systems.



> docs/

I assume you mean Documentation.


> docs/learning.tely
> docs/learning/*.itely
> docs/notation.tely
> docs/notation/*.itely
> + glossary, essay, application

Fine, but Info file names must be named lilypond (for the main Info
documentation, probably some kind of part of the web site) or
lilypond-*. IIRC source file names don't have to have the same file name
prefix as the Info file, I'll have to check this.


> docs/snippets.tely    (moved from input/lsr/ and input/new/)
> docs/snippets/*.ly
> 
> docs/contributors.texi
> docs/contributors/*.texi
> 
> docs/web.texi
> docs/web/*.itexi

Why not, but this can only be almost permanent contents IMHO (I'll try
to explain at the end of this message).


> docs/topdocs/
> docs/topdocs/changes.texi

You mean changes.tely, don't you?


> docs/topdocs/compile.texi

Do you mean this would be a standalone document?


> regression/   new location of input/regressions/
> input/        completely deleted

Can somebody already starts checking where could go stuff that remains
in input/{tutorial,manual}?



> How much cross-referencing do we want?  Probably not a huge
> amount, but enough that is could be a pain.  Web->Manuals links to
> every manual, of course.  The FAQ -- which might live as part of
> manuals.itexi in manuals, or as a separate manual in docs/ --
> wants to link to the LM, website, and AU.  Quite possibly other
> items, too.  The AU will probably want to link to certain pages on
> the website.  LM probably also wants to link to parts of the
> website (Community->Contact ->Tiny examples, and maybe ->Bug
> reports).

Speaking of cross-references, I'll add sooner or later a makefile hook
that runs the cross-reference checking script I wrote and/or some
Texi2html-internal hook, which will break the doc build if there are any
broken cros-references (including cross-manuals cross-references). We
are not ready to be so strict now because of the web+docs upcoming
overhaul, but so much effort has been put into documentation that this
important quality criterion is checked.




> There are other considerations: I'd like to include the authors
> and THANKS file in Community->Authors.  I suppose we could move
> all the author bios into a separate branch... but what about the
> THANKS file?  I _guess_ we could hack up a git checkout, so
> whenever you built the docs in a separate web/ repository, it
> would get the latest THANKS from the separate lilypond.git
> repository... but this sounds ridiculous to me.
> 
> Ditto for the Changes manual (the renamed NEWS) and FAQ.  The
> master copy of this obviously needs to be in the source, but that
> means that we either distribute these as fully distinct manuals (a
> one-page lilypond-faq.pdf ?), or do some weird hackery to include
> it in the build process of a separate website repository.

In all that reasoning, you missed the essential fact that as long as
we'll have two branches, building the web site will require at least two
branches (or by-products of two branches), one for the development
branch, the other for the stable branch; (BTW you pointed this out later
in your message). So some "weird hackery" is needed to get (download or
cd into another local Git branch and generate) generated Texinfo xref
maps from the other branch to generate correct HTML links, or
alternatively we are bound to have the same node names and output
directories in both branches.

This also raises the web site build host issue: a hourly cron job on
lilypond.org for building is currently incompatible with having ly
snippets compiled during the build; even if we can get Lily to run
safely, I doubt that lilypond.org host will provide efficiently the
needed CPU and memory ressources while keeping its server role.


> I know that there's some concerns about people accidently screwing
> up the website and having it show up in an hour.  I've never been
> quite certain why we have an automatic hourly rebuild -- we don't
> update the website all that often.  We could disable that for a
> few months, see how often people screw up the website and go with
> a script to upload the generated website, then re-evaluate the
> situation.

The hourly script is useful to allow any web site maintainer to react
quickly against typos and minor bugs that don't require fixing the
Makefile or Python scripts, whereas it could take much longer if you (or
another maintainer) are the only person that can fix problems. I'm not
keen on dropping the hourly build unless there are two or three active
web site maintainers (i.e. people that have a SSH access to Lily main
web site). The cron hourly job is also handy to quickly publish news
items. BTW I'm sure Valentin is tempted to say (and I'd agree with him)
that lilypond.org should be a bit more dynamic than it is currently,
suppressing the hourly build would not go in the right direction.

If the web site in the documentation is completely integrated into the
source tree, maintaining the hourly build could be cumbersome for
lilypond.org maintainer; if we factorize the makefiles used to build the
documentation and the website (which is the only reason I see to merge
the web site into the docs as much as possible), the maintainer will
have to frequently update a significant number of makefiles, and you
might find the initial setup for this quite painful (creating a
"trusted-scripts" copy etc.). I hope it's possible to find a compromise,
again please see at the end.


> I guess we could have a different makefile target to specify
> building the web/ as the main website, versus building web/ as a
> local info, pdf, or html file.  In "local" mode, the links in
> Manuals would point to whatever the rest of the docs were, whereas
> in "web" mode, they'd point to the appropriate stable/devel
> versions.

This might be doable, but I even think there should be a hourly building
routine lilypond.org, that would rely on already built and uploaded docs
and parts of the website, but that needn't the whole makefile armada of
Lily sources nor calling lilypond.


> Whatever.  I think the documentation/usage goal of more
> integration between the docs and website is sound.  I'm now less
> certain about the best technical implementation of that idea, but
> as long as the usage goal is met, I won't complain any more.

...unless you keep being persecuted for uploading the web site or
updating makefiles on lilypond.org :-P

***

Now, please let me modify your initial proposal and give shape to some
technical details.  I think of splitting the web site source/build in a
temporal part and an atemporal part (I stole this idea from Paris 6
University Computer Science Teaching Department web site, but in a
biased way, as they have two rather distinct web sites).

The temporal part would be contents that changes often and/or that
requires information from several branches: the news and the download
pages. Building it would require an active Internet connection, a read
access to Lily git repo and a build of the website including some
by-products (e.g. xref-map files to get cross-references right).

The atemporal part would be all the rest, which would be compiled from
master branch sources in Documentation/web and would follow the usual
release cycle, allowing verification, validation, translation updating
between releases.

The offline version of the web site would be available in the docball,
temporal pages would be replaced by atemporal flavors of these ones.

I'm not sure about where the temporal part should live; the simplest
would be keeping it in web branch, so we are sure not to mix technical
differences between them.


Please comment,
John