GOP-PROP 5: build system output (update)

[Graham Percival]

Update on this; I'm not ready to call it a "probable decision"
yet.
http://lilypond.org/~graham/gop/gop_5.html

(proposal written by Phil Holmes, modified by Graham)


** Proposal summary

When you run make or make doc,

    * All output will be saved to various log files. (including
      output from make(1))
    * We will still display the output of make(1) on the console.
    * No other output will be displayed on the console, with one
      exception: if a build fails, we might display some
      portion(s) of log file(s) which give useful clues about the
      reason for the failure.
    * Logfiles from calling lilypond (as part of lilypond-book)
      will go in the relevant
      ‘build/out/lybook-db/12/lily-123456.log’ file. All other
      logfiles will go in the ‘build/logfiles/’ directory.
    * Both stderr and stdout will be saved in *.log. The order of
      lines from these streams must be preserved.
    * There will be no additional “progress messages” during the
      build process. If you run make --silent, a non-failing build
      should print absolutely nothing to the screen.
    * Ideally, a failing build should provide hints about the
      reason why it failed, or at least hints about which log
      file(s) to examine. 


** Rationale

Before any of the current work on reducing output from make, the
result of a "make doc" was over 500,000 lines of text. Finding
errors or warnings in that volume of output is always going to be
hard. The prime reason for the output being so verbose is that all
the processes that run as a result of the call to make echo their
output to the screen, often in verbose mode. Lilypond itself
produces around 370,000 lines of output as a result of
lilypond-book building all the snippets.

Much of this output can be redirected to logfiles and so the
impossible-to-read clutter on the screen is cut out and could be
referred to later. However, there is a danger in this approach,
that vital error messages can also be lost, thus preventing the
cause of the failure of a make being found. We therefore need to
be exceptionally careful to move cautiously, include plenty of
tests, and give time for people to experiment/find problems in
each stage before proceeding to the next stage.

A variable, QUIET_BUILD, can be set and this will reduce the
clutter but not eliminate it. (see
http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables
) This variable currently does things like adding a -q flag to the
TEXI2PDF call ("quiet") and getting rid of the –verbose flag in
some calls to LilyPond. However, it could be used more widely, as
proposed below.


** Implementation notes

none yet 


Cheers,
- Graham