Re: GOP-PROP 9: behavior of make doc (probable decision)

[Janek Warchoł]

Hm,

LGTM?

Phil, i'm amazed by your patience.
thanks,
Janek

2011/8/17 Graham Percival <address@hidden>:
> Not much discussion, not much change from the last version.
>
> http://lilypond.org/~graham/gop/gop_9.html
>
>
> Proposal summary
>
> If there are build problems, then it should be easier to find out
> why it’s failing. This will be achieved with log files, as well as
> possibly including scripts which automatically display portions of
> those log files for a failing build.
>
> We will also add targets for building a specific manual (for
> quick+easy checking of doc work), as well as for building all
> documentation in a specific language (either English or a
> translated language).
> Rationale
>
> When the lilypond doc build breaks, it’s too hard to figure out
> why it broke.
>
> We see emails to lilypond-devel approximately once every four
> months about broken doc builds. On a subjective note, Graham has
> been the documentation editor since 2003, but even he cannot
> reliably pinpoint the cause of a failing doc build within 10
> minutes. We waste a ridiculous amount of time, effort, and
> patience on doc build problems.
> Sea of output
>
> Before any of the current work on reducing output from make, the
> result of a “make doc” was over 500,000 lines of text. The prime
> reason for the output being so verbose is that all the processes
> that run as a result of the call to make echo their output to the
> screen, often in verbose mode. Lilypond itself produces around
> 370,000 lines of output as a result of lilypond-book building all
> the snippets.
>
> Much of this output can be redirected to logfiles and so the
> impossible-to-read clutter on the screen is cut out and could be
> referred to later.
> Proposal details
>
> When you run make doc,
>
>    * All output will be saved to various log files, with the
>    * exception of output directly from make(1).
>
>      Note that make(1) refers to a specific executable file on
> unix computers, and is not a general term for the build system.
>    * By default, no other output will be displayed on the
>    * console, with one exception: if a build fails, we might
>    * display some portion(s) of log file(s) which give useful
>    * clues about the reason for the failure.
>
>      The user may optionally request additional output to be
> printed; this is controlled with the VERBOSE=x flag. In such
> cases, all output will still be written to log files; the console
> output is strictly additional to the log files.
>    * Logfiles from calling lilypond (as part of lilypond-book)
>    * will go in the relevant
>    * ‘build/out/lybook-db/12/lily-123456.log’ file. All other
>    * logfiles will go in the ‘build/logfiles/’ directory.
>
>      A single make doc will therefore result in hundreds of log
> files. Log files produced from individual lilypond runs are not
> under our control; apart from that, I anticipate having one or two
> dozen log files. As long as it is clear which log file is
> associated with which operation(s), I think this is entirely
> appropriate. The precise implementation will be discussed for
> specific patches as they appear.
>    * Both stderr and stdout will be saved in *.log. The order of
>    * lines from these streams should be preserved.
>    * There will be no additional “progress messages” during the
>    * build process. If you run make --silent, a non-failing build
>    * should print absolutely nothing to the screen.
>    * Assuming that the loglevels patch is accepted, lilypond
>    * (inside lilypond-book) will be run with –loglevel=WARN.
>    * http://codereview.appspot.com/4822055/
>    * Ideally, a failing build should provide hints about the
>    * reason why it failed, or at least hints about which log
>    * file(s) to examine.
>
> If this proposal is accepted, none of these policies will be
> assumed to apply to any other aspect of the build system. Policies
> for any other aspect of the build system will be discussed in
> separate proposals.
> Don’t cause more build problems
>
> However, there is a danger in this approach, that vital error
> messages can also be lost, thus preventing the cause of the
> failure of a make being found. We therefore need to be
> exceptionally careful to move cautiously, include plenty of tests,
> and give time for people to experiment/find problems in each stage
> before proceeding to the next stage.
>
> This will be done by starting from individual lilypond calls
> within lilypond-book, and slowly moving to “larger” targets of the
> build system – after the individual lilypond calls are are
> producing the appropriate amount of output and this is saved in
> the right place and we can automatically isolate parts of a
> failing build, we will work on lilypond-book in general, and only
> then will we look at the build system itself.
> Implementation notes
>
> There is an existing make variable QUIET_BUILD, which alter the
> amount of output being displayed
> (http://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables
> ). We are not planning on keeping this make variable.
>
> The standard way for GNU packages to give more output is with a
> V=x option. Presumably this is done by increasing x? If we support
> this option, we should still write log files; we would simply
> print more of the info in those log files to screen.
>
> The command tee may be useful to write to a file and display to
> stdout (in the case of VERBOSE).
>
>
> Cheers,
> - Graham
>
> _______________________________________________
> lilypond-devel mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/lilypond-devel
>