Re: [Groff] More referenceability for -mdoc would be an improvement!?

[Steffen Nurpmeso]

Hallo Ingo,

Ingo Schwarze <address@hidden> wrote:
 |Steffen Nurpmeso wrote on Sat, Sep 13, 2014 at 05:07:27PM +0200:

 |> So i really like your .Ix multiplexer idea.
 |
 |Not really a multiplexer, it's just giving one single name to one
 |single, logically self-contained piece of functionality.

Sure thing.  Let it be a multiplexer for me :)

 |>   address@hidden src]$ mandoc -Tlint x.1
 |>   mandoc: x.1:44:22: WARNING: undefined string, using "": Ix
 |>   mandoc: x.1:44:9: WARNING: undefined string, using "": Ix
 |>   mandoc: x.1:48:2: ERROR: skipping unknown macro: .Ix Ev dummy
 |
 |That doesn't really matter.
 |
 |I didn't say old tools should be able to process manuals using
 |new features without warnings and error messages.  I only meant
 |they should still produce usable output, if possible.
 |
 |Actually it's good if they warn.  That tells the user that the
 |document uses features the tools don't support.  From the mandoc(1)
 |perspective, ERROR is better here than WARNING because there is a
 |risk of significant information loss (even though what remains is
 |probably still about as good as old documents not using the
 |functionality in the first place).  But seeing just a warning would
 |not really be a problem, either.

Yes.  Yes.  That actually sounds good, but i'm nonetheless happy
that the default behaviour is to be silent and not cause any
output mutations!

 [.]
 |> So, and why i'm writing this, i think that, for X-roff,
 |> it won't be accepted if this mode of operation would be made the
 |> default.
 |
 |I don't think any mode switches will be acceptable in this context.
 |It should just work.  People don't want to twist knobs all day,
 |they merely want to read documentation.

 |> It will require a "complete rewrite" of the TTY device, for
 |> example,
 |
 |I'm not yet convinced anything meaningful can be done for terminal
 |output.  I guess in the first step, groff will merely parse it
 |without complaint, in the second, it will be mostly for HTML,
 |in the third step for PDF, and *maybe* someday for ASCII.  We
 |certainly shouldn't start with ASCII.
 |
 |> which is yet sequential but then has to wait until EOF
 |> just to lookout wether any .Ix anchor occurred.
 |
 |Encoding the link destinations (anchors) in ASCII at all is
 |already a huge problem, the problem of forward references is
 |minor compared to that and allows several different solutions.
 |
 |> Of course a new \X'' has to pass additional info through etc.
 |> Etc. etc.  (Symbol table etc.)
 |
 |Right now, doc.tmac doesn't do more than plain Sy and Em font
 |instructions even for .Lk.  Not sure we have to aim higher than
 |that for the first version.
 |
 |> Short: i think a second command / string / number register has to
 |> be defined that turns on the new functionality.  If set, mdoc (+)
 |> will emit \X'' sequences and the driver would take appropriate
 |> steps for the two-pass mode.
 |
 |I'm not even convinced two-pass is needed.  Certainly not at the
 |beginning, and maybe never.  Forward references will be the
 |exception rather than the rule.  So one obvious shortcut is to
 |simply ignore forward references if they cause trouble and only
 |handle backward ones.  Another might be to simply handle all newly
 |occurring, non-anchored content macros as forward references; as
 |everything ought to be handled *somewhere*, that's likely to just
 |work.  One could also allow forward declarations, allowing authors
 |to state "this is a forward reference".  Lots of ways, not at all
 |sure which one will turn out to be the best.

Well so-so, hm, unfortunately and no, respectively.
Let me tell you what i'll do.
For S-roff i'm still in the stage of adjusting the copyrights,
making actual program names etc. configurable and print what has
been requested etc., which is a terribly boring and brain-dead
task -- i'm afraid i would loose any passion if i would
temporarily switch to something of actual interest.

So next week (hopefully) i will sit down and write a patch against
current -mdoc that implements an .Ix stub and documents it, adding
an entry to the bug tracker.
I think .Ix should at least support referencing of Dv, Ev, Fn, Ic,
Va, printing an error message for other uses.  Does this sound
right / sufficient / acceptable to mandoc(1)?

Since the improved referencing / indexing capabilities will refer
to new or actively updated manuals only i think it is acceptable
to require that one of the first commands be an ".Ix 1" to
indicate that the document makes use of the extensions -- this
would affect the -mandoc mode of mandoc(1) as that currently
doesn't detect the right macro set when .Ix comes first (instead
of .Dd, .Dt and ...), yet i think placing it first would be a
logical decision from a manual writers point of view.
(Unfortunately mdoc doesn't define a request that indicates a
documents' version or required feature set (like e.g. perl(1)s
$^UNICODE) which could be used / added to for that.  And from my
point of view adding one now is complete overkill.)

I think any non-".Ix 1" that occurs without having seen the
unlocking ".Ix 1" first should print an error message, too.

So consensus on the above is of course a prerequisite.

I'd also look into less(1) and write a proof-of-concept patch.
(Again, the idea is to have a simple command shortcut, say, ^X,
read in a decimal number, expand that to a backspace sequence and
start a simple search.)

Later on, hopefully this year, i'll adjust the -- then usable --
S-roff TTY mode to support the two pass model that is necessary to
be able to track all references -- one cannot assume that all .Dv,
.Ev etc. that are used throughout a manual are actually defined in
the same manual, right?  I assume mandoc(1) can simply traverse
it's tree...  I'll add to the bug report a corresponding patch for
grotty, then, including the necessary changes to fill the stubs of
-mdoc.

I'm pretty sure that once people look at a manual page in $PAGER
and have the option to simply enter a number to jump to the place
where the symbol in question is defined they don't want to miss it
again.  Not to talk about -Thtml / -Txhtml output of manuals that
actually provides the real value of hyperrefs.
And being able to generate automatic indices is hammer.  I didn't
think about this initially and on my own!
Thanks, Ingo!

P.S.: what about a ".To [number]" request that embeds an
automatically created table of contents for all .Sh and .Ss of
a document?  That would also be a real win, but this time
_especially_ for HTML and PDF output..

--steffen