Re: make check is broken (again) - patch testing seeming to taking more of my time than I like

[Werner LEMBERG]

> In the past month, I've devoted many hours to testing my
> submissions, but clearly the effort is not achieving the goal.

Don't worry.  Some of the problems are very hard to catch and only
show up under certain circumstances.

> I request some help to understand how I can improve my pre-commit
> testing procedures, and where my responsibilities begin and end.

IMHO, you did well.

> I enjoy having my commits reverted as much as others enjoy having
> their build broken--it is a big waste of pro-bono time--so I want to
> understand the issues clearly.

There is no waste of time, since the reversion is only temporary –
your patches are definitely not rejected.  However, small amendments
are apparently necessary to make them work everywhere, and the
problems were caught unusually late.

> And days before that happens, patches are announced as having been
> tested with the feedback "Passes make. make check and a full make
> doc."  The evidence suggests that that does not include running
> autogen, otherwise it should have caught the problem with "tidy"
> that my own testing failed to catch.

Yes.  A full run of everything is not always executed.

> Should things such as missing optional programs and new-ish Python
> syntax be rejected at either of these stages?  If not, then it would
> seem to fall to the submitter to set up an alternate development
> environment with Python 2.4, GCC 3.4, and similarly aged versions of
> other tools, and run additional tests in that environment.

A good test for those things IMHO is to clone the gub repository, then
saying

  make lilypond

The first run takes many hours to download and build the necessary
infrastructure; later on it's much faster.  Somewhere in the
lilypond-devel mailing list archive you can find hints how to make gub
use your own clone of the git repository so that you can directly test
patches; ditto for hints that explain what created stuff in gub should
be manually removed in case you want to test a full lilypond build
(without unnecessarily rebuilding the gub infrastructure).

Apropos gub: Inspite of David K's reversions there is still a (new?)
bug in `output-distance.py' (I think): The call

  PYTHONPATH=[...] \
  PATH=[...] \
  python2 test-lily/rsync-test.py \
    --version-file=... \
    --output-distance=.../scripts/build/output-distance.py \
    --test-dir=uploads/webtest \
    --gub-target-dir=...

in the `unlocked-test-export' rule fails with

  [...]
  entering directory .../gub/uploads/webtest/v2.21.0-1-unpack/v2.21.0-1
  invoking
    gs -sDEVICE=png16m \
       [...] \
       
-slilypond-datadir=.../gub/uploads/webtest/v2.21.0-1-unpack/v2.21.0-1/share/lilypond/current
 \
       [...] \
       rest-positioning.eps \
       -c quit

  Error: /undefinedfilename in --file--
  Operand stack:
    
(.../gub/uploads/webtest/v2.21.0-1-unpack/v2.21.0-1/share/lilypond/current/fonts/otf/emmentaler-20.otf)
   (r)
  Execution stack: [...]
  Last OS error: No such file or directory

because the `share' tree present in

  gub/uploads/webtest/v2.21.0-1-unpack/v2.19.83-1/

is not created by the script in

  gub/uploads/webtest/v2.21.0-1-unpack/v2.21.0-1/

I seem to remember that this has worked the last time I worked on gub
(in January) ...


    Werner