Re: two further apparent regressions in 1.23 in s.tmac

[G. Branden Robinson]

[replying only to the lists, since you're not honoring the Reply-To
header anyway]

Hi Joerg,

At 2024-10-16T22:01:02+0200, joerg van den hoff wrote:
> > At 2024-10-16T13:30:23+0200, joerg van den hoff wrote:
> > > I ran into this problem several months ago but did never suspect
> > > it would be related to a preceding update to 1.23 (which happened
> > > as part of general package update run so I did not even take note
> > > until recently that groff on my machine is actually now at 1.23)
> > > so this report comes a bit late.
> > 
> > Did you review the NEWS file or the release announcement to see if
> > this issue was covered there?
> > 
> > https://git.savannah.gnu.org/cgit/groff.git/tree/NEWS?h=1.23.0
> > https://lists.gnu.org/archive/html/info-gnu/2023-07/msg00001.html
> 
> no. I considered this an obvious bug.

I see.  The views of the developers have no weight to you, even though
it is they who maintain the software.

> I had no incentive to read the NEWS in this context.

What _would_ incentivize you to read a software package's release notes?

> > What is this `rt` doing without a preceding `mk`?
> 
> my understanding is that w/o mk the default is for rt to return to
> page top.  at least that is what it empirically has done for the last
> 25 years I believe.
> 
> I now double-checked: the manual seems to indicate that w/o preceding
> mk, rt needs an argument (distance from page top).

Yes.  I'd say "seems" is an understatement.

groff(7) [from groff Git][1]:
     .rt        Return (upward only) to vertical position marked by .mk
                on the current page.
     .rt N      Return (upward only) to vertical position N (default
                scaling unit v).

Here's our Texinfo manual (from 1.22.4) on the same subject:

5.22 Page Motions
=================

*Note Manipulating Spacing::, for a discussion of the main request for
vertical motion, 'sp'.

 -- Request: .mk [reg]
 -- Request: .rt [dist]
     The request 'mk' can be used to mark a location on a page, for
     movement to later.  This request takes a register name as an
     argument in which to store the current page location.  With no
     argument it stores the location in an internal register.  The
     results of this can be used later by the 'rt' or the 'sp' request
     (or the '\v' escape).

     The 'rt' request returns _upwards_ to the location marked with the
     last 'mk' request.  If used with an argument, return to a position
     which distance from the top of the page is DIST (no previous call
     to 'mk' is necessary in this case).  Default scaling indicator is
     'v'.

> if it _really_ does the line would have to read `.rt 0'. but doing
> this does not change behaviour neither for 1.22 nor for 1.23: AFAICS
> the resulting pdfs are unaltered.
> 
> so apparently `.rt' w/o argument and preceding `mk' indeed is valid
> and _does_ (and might prospectively be guaranteed to) return to page
> top. if so, the groff manual regarding .rt/.mk should then mention
> this circumstance.

groff's Texinfo manual goes on to say:

     If a page break occurs between a 'mk' request and its matching 'rt'
     request, the 'rt' is silently ignored.

> if it _really_ does the line would have to read `.rt 0'. but doing
> this does not change behaviour neither for 1.22 nor for 1.23: AFAICS
> the resulting pdfs are unaltered.
>
> so apparently `.rt' w/o argument and preceding `mk' indeed is valid
> and _does_ (and might prospectively be guaranteed to) return to page
> top. if so, the groff manual regarding .rt/.mk should then mention
> this circumstance.

It's not "valid": it's undefined.[2]  You relied on undefined behavior
but got an outcome you were happy with, and have proceeded to elevate
that behavior to the status of specification, apparently because it
satisfies your preference.

This is of a piece with your other observations, a point to which I
return below.

> > Would you please follow-up with a complete document as a reproducer?
> > It can still be "minimal" insofar as it reproduces the problem(s).
> 
> I do not understand. the snippet I sent can be considered "the
> document". to make it compliant with currently documented behaviour
> replace `.rt' by `.rt 0'.

Your snippet/document was exercising undefined behavior (or relying upon
`ms` internals to set the anonymous vertical position mark _for_ you),
so I suspected that you might have elided more from your snippet than
you intended.

I understand now that you are content to exercise any unspecified
formatter or macro package behavior you can identify, pour cement on it,
and decree any alteration of such unspecified behavior as a "bug".

On many machine architectures, in the C language, a pointer occupies the
same number of bits as an `int` variable.  Clearly we can assume that
they're always equivalent; if our code works despite some tedious manual
warning us not to rely on this conclusion, we should do so anyway.

> > The release notes/NEWS file may be helpful here.
> 
> I do not understand. if you think the reported behaviour (the spacing
> issue) and the difference to 1.22 is _not_ a bug, I would appreciate a
> short hint where exactly _that_ is resolved in the 17 pages NEWS...

Okay.  Here you go.

https://git.savannah.gnu.org/cgit/groff.git/tree/NEWS?h=1.23.0#n426

> if it _is_ a bug (I would definitely think it is since, again, it
> breaks 25+ years of consistent groff behaviour),

How do you know?  Have you run any tests with groff 1.12.1 lately?

> [...]I will not find it in the NEWS.  I have reported it in the hope
> that it is of interest to you.

I'm taking note of it as reinforcing my impression that Berkeley's
introduction of the `DD` register feature was under-specified.

> if not so much because you think it is not a bug, too bad.

It's often difficult to please everyone simultaneously when
unspecified/undefined behavior is at issue.  ISO/IEC JTC1/SC22/WG14 has
much experience in this respect.

> thanks for looking into the issue. as said, I have reverted to 1.22
> since I currently need to get some writing/formatting done and don't
> have much time for further 1.23-related delays/distractions right now.

Okay.

> but I trust you have tested your fix yourself...

It's possible.

https://git.savannah.gnu.org/cgit/groff.git/commit/?id=bc0103ddd08966578989d717f31cd1f273a9544f

Since my Carnac hat predicts that, if you follow the link, you'll read
the words "Test fails at this commit" and leap to a conclusion that I
haven't fixed the problem, here's another URL.

https://git.savannah.gnu.org/cgit/groff.git/commit/?id=7fbda04469c0581e7d6394291ca3d1c13483c3ed

> > No point release to groff 1.23 is anticipated.  The next release is
> > expected to be 1.24.  To get a preview of what user-visible changes
> > to ms are anticipated in the next release, you can consult the NEWS
> > file in our Git repository.
> 
> your decision. in my opinion the recently surfaced regression (the .XA
> issue) alone would deserve that since otherwise it will take
> substantially more time until it is officially fixed with releasing
> 1.24.

I hope to release 1.24 before the end of the calendar year.  I have some
years of familiarity with the stable package update practices of the
Debian Project, and my inference is that the fix for the `XA` bug comes
nowhere near the threshold for such a backport.  I also seem to recall
that the consensus of the groff mailing list, reached some years ago, is
that our team is not large enough to support multiple branches of
development.  Our headcount has not increased substantially since then.

This isn't to say that you couldn't petition Debian, Red Hat, Arch, and
other Linux distributors to recalibrate their backport/stable update
policy to more closely conform to your perturbation threshold.

As it happens, Debian's groff package maintainer, Colin Watson, has been
a fixture of the groff list for years, and has made a couple of dozen
commits to the repository over time.  He's well-placed to gainsay me.

> > [1] The lack of `br` meant that a document that started with a `DS`
> >     macro call did not "sweep" the drawing position past vertical
> >     position 0, which springs a trap that calls a macro called
> >     `cov*first-page-init`,[2] which does much essential
> >     initialization, including (indirectly) setup of the `PS`
> >     register storing the type size.  Consequently, the text of the
> >     display would render at 1 point(!) instead of the formatter's
> >     default of 10.  The diagnostic about the `nf` environment might
> >     warn the user that the formatting is likely to go awry, but we
> >     can do better than that by simply restoring the break.
> 
> IIUC ("restoring the break"), this is the same sort of regression as
> with the .XA macro?

That seems a fair assessment.

> deleting a "superfluous" line of code as a "code tidyup"?

I winced when I saw it; it was mixed in with an improvement in argument
validation and diagnostic reporting.

> if so, I would rate this as an indication that you should consider to
> increase your personal "confidence threshold" which you require to be
> reached before you proceed with such an action.

Next time I have a time machine to go back three years and exhort myself
to think twice, I'll surely take your advice.

> as said previously, the working hypothesis "original author might be
> smarter (at least not less smart) than me and had a reason to put that
> line in" could help.

The use of manifest program source code as a proxy measure of
intelligence is noteworthy for how many problems it's caused among
software engineers over the decades.

One might observe that the "smart" developer writes automated tests[3]
to measure the behavior of a complex software system, so that the
multifarious interactions of its components remain within expected
bounds on a mechanically (and independently, externally) verifiable
basis instead of resting solely upon the intelligence or mastery level
of an authoring developer, who might retire, lose their mental
faculties or interest in the project, or otherwise become unavailable.

By _that_ metric, the intelligence level of groff releases through
1.22.3 was minimal (only the "gdiffmk" program had any tests).  For the
ms package in particular, it was at zero for the entire lifespan of the
1.22 series.

A system that depends solely upon the intelligence of its maintenance
personnel is a fragile one, not a robust one.  Have you ever talked to
a pilot or an electrical lineworker about the nature of their job, and
the rules and safety measures that are part of it?

I appreciate your candor--your perspective enables me to conclusively
decide the matter of what sort of person I'm dealing with, a question
whose answer you had only suggestively hinted toward before.

> that's my approach with other peoples code at least when I adjust it
> to my needs.

I conjecture that you've had better luck with the code people have
developed than with the people themselves, if you so speedily and
reflexively resort to derogating and insulting their intelligence.

Regards,
Branden

[1] groff(7) from 1.22.4 differs little:

     .rt       Return (upward only) to vertical position marked by .mk
               on the current page.
     .rt ±N    Return (upward only) to specified distance from the top
               of the page (default scaling indicator v).

[2] CSTR #54 has equivalent language, not specifying the behavior of an
    argumentless `rt` request when `mk` has not been called while
    formatting the same page, but I leave verification of this
    equivalence to the reader.

[3] or machine-verifiable proofs in a formal verification system

signature.asc
Description: PGP signature