Re: Another vertical page layout issue

[Joe Neeman]

On Fri, 2010-01-15 at 09:06 +0100, David Kastrup wrote:
> From the reports so far I get the impression that the new vertical
> layout code is fundamentally broken, and the last fixes to it have
> mostly been skirting the issue, in the class of "I don't know why this
> would happen, but we can fix it this way".  As more and more utterly
> wrong cases get patched away, the issue gets hidden more and more.  The
> result being wrong decisions that get ameliorated by "fixes" until the
> most glaring cases are masked.

I don't believe this is the case (although of course I am biased). First
of all, I'd like to distinguish between the page-breaking code (which
has been around for a few stable releases now) and the vertical layout
code (which doesn't determine the page breaks, but merely the vertical
positioning on each page).

The main issue with the page-breaking code is that finding the actual
outlines of systems is too expensive, so we use estimates which aren't
(and will probably never be) perfect. The recent overfull-page issues
stem from the fix for 496 (which fixed a height overestimation issue and
therefore exposed some height underestimation issues). 4906be555c is an
exception: it was caused by a somewhat subtle bug that had been around
for a while. But the fix made sense, in that it fixed a genuine bug
rather than papering over something.

> When the real problem may be something like "<" being used somewhere
> instead of ">", or a similar problem.
> 
> Joe, would you say that the bad cases you fixed so far had "a right" to
> occur before your fixes, in that one could naturally see how they would
> arise as a consequence of the chosen algorithm and its design goals?

I can't think of such a case, except possibly for Werner's example
"strange vertical formatting" on lily-devel, which we agreed was a
strange corner case that didn't need fixing (only a warning).

> If it's not the case, there is an implementation problem, and patching
> around locally is going to make it harder to understand the code and to
> find the real issue.

That depends. If the "local" patches fix mistakes without adding
complexity, I don't think they make the code harder to understand. I
believe that my last 3 patches to the page layout code (0f9645def2,
fcfbd212006 and 2bed451f9a) fall under this category.

> I've written global page break optimization code (in the context of TeX
> programming) myself, and gone through similar experiences where I
> recoded and recoded and a rather glaring sign error that should have
> wrecked things rather obviously managed to stay around for at least a
> year because it was masked in many cases.

There may well be sign errors in the current code (there certainly were
some previously in the new layout code). However, I don't think there is
a single, glaring mistake underlying the recent page-layout bugs since I
don't believe that I've been pushing any "mysterious,"
"covering-things-up" kind of patches. But if you find examples, I'm
happy to be corrected.

Joe