GOP-PROP 9: behavior of make doc (probable decision)

[Graham Percival]

Not much discussion, not much change from the last version.

http://lilypond.org/~graham/gop/gop_9.html


Proposal summary

If there are build problems, then it should be easier to find out
why it’s failing. This will be achieved with log files, as well as
possibly including scripts which automatically display portions of
those log files for a failing build.

We will also add targets for building a specific manual (for
quick+easy checking of doc work), as well as for building all
documentation in a specific language (either English or a
translated language).
Rationale

When the lilypond doc build breaks, it’s too hard to figure out
why it broke.

We see emails to lilypond-devel approximately once every four
months about broken doc builds. On a subjective note, Graham has
been the documentation editor since 2003, but even he cannot
reliably pinpoint the cause of a failing doc build within 10
minutes. We waste a ridiculous amount of time, effort, and
patience on doc build problems.
Sea of output

Before any of the current work on reducing output from make, the
result of a “make doc” was over 500,000 lines of text. The prime
reason for the output being so verbose is that all the processes
that run as a result of the call to make echo their output to the
screen, often in verbose mode. Lilypond itself produces around
370,000 lines of output as a result of lilypond-book building all
the snippets.

Much of this output can be redirected to logfiles and so the
impossible-to-read clutter on the screen is cut out and could be
referred to later.
Proposal details

When you run make doc,

    * All output will be saved to various log files, with the
    * exception of output directly from make(1).

      Note that make(1) refers to a specific executable file on
unix computers, and is not a general term for the build system.
    * By default, no other output will be displayed on the
    * console, with one exception: if a build fails, we might
    * display some portion(s) of log file(s) which give useful
    * clues about the reason for the failure.

      The user may optionally request additional output to be
printed; this is controlled with the VERBOSE=x flag. In such
cases, all output will still be written to log files; the console
output is strictly additional to the log files.
    * Logfiles from calling lilypond (as part of lilypond-book)
    * will go in the relevant
    * ‘build/out/lybook-db/12/lily-123456.log’ file. All other
    * logfiles will go in the ‘build/logfiles/’ directory.

      A single make doc will therefore result in hundreds of log
files. Log files produced from individual lilypond runs are not
under our control; apart from that, I anticipate having one or two
dozen log files. As long as it is clear which log file is
associated with which operation(s), I think this is entirely
appropriate. The precise implementation will be discussed for
specific patches as they appear.
    * Both stderr and stdout will be saved in *.log. The order of
    * lines from these streams should be preserved.
    * There will be no additional “progress messages” during the
    * build process. If you run make --silent, a non-failing build
    * should print absolutely nothing to the screen.
    * Assuming that the loglevels patch is accepted, lilypond
    * (inside lilypond-book) will be run with –loglevel=WARN.
    * http://codereview.appspot.com/4822055/
    * Ideally, a failing build should provide hints about the
    * reason why it failed, or at least hints about which log
    * file(s) to examine. 

If this proposal is accepted, none of these policies will be
assumed to apply to any other aspect of the build system. Policies
for any other aspect of the build system will be discussed in
separate proposals.
Don’t cause more build problems

However, there is a danger in this approach, that vital error
messages can also be lost, thus preventing the cause of the
failure of a make being found. We therefore need to be
exceptionally careful to move cautiously, include plenty of tests,
and give time for people to experiment/find problems in each stage
before proceeding to the next stage.

This will be done by starting from individual lilypond calls
within lilypond-book, and slowly moving to “larger” targets of the
build system – after the individual lilypond calls are are
producing the appropriate amount of output and this is saved in
the right place and we can automatically isolate parts of a
failing build, we will work on lilypond-book in general, and only
then will we look at the build system itself.
Implementation notes

There is an existing make variable QUIET_BUILD, which alter the
amount of output being displayed
(http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables
). We are not planning on keeping this make variable.

The standard way for GNU packages to give more output is with a
V=x option. Presumably this is done by increasing x? If we support
this option, we should still write log files; we would simply
print more of the info in those log files to screen.

The command tee may be useful to write to a file and display to
stdout (in the case of VERBOSE).


Cheers,
- Graham