Re: [Groff] About numerical precision

[Thomas Baruchel]

[answering to Ted and Ralph]
>   .nr Y 36432
>   .nr X 46800
>   \s[72]
>   .nr N \nY*\n(.s/\nX
>   \nN
Yes, wonder why I had an overflow, but having at least 1 correct
decimal is an important issue for me. Of course, I wrote the macro
making my tests with \s[72], but the real usage will be \s[12].
I know I can have exactly what I want for the horizontal .9 scale,
but \H'10' isn't in my opinion the same than \H'10.5', since my whole
approach is micro-typography. It is true that for \s[72], \H'56' isn't
bad (the real value would be 56.049, and i would also tell it isn't bad
for rounding 56.5 to 56), but 10 instead 10.5 isn't at all what I want.

Here is my file

.fam MO
.de SMALLCAPS
.nr x \\w'x'-\\w'x'+\\n[rst] \" 31248 x regular
.nr X \\w'X'-\\w'X'+\\n[rst] \" 46800 X regular
.nr x (\\nX-\\nx)/3+\\nx     \" 36432 X smallcaps
\H'\\nx*\\n(.s/\\nX'\s[9*\\n(.s/10]test
..
.sp 72p
.ps 72
.SMALLCAPS M OZART

(of course, the macro isn't finished, and the arguments aren't handled yet).
I try it with postscript device 1z = 1000u
I get a multiplication overflow at the \H'' line, expecting a real number
with 1 or 2 correct decimals (Yes, I know 56 isn't bad for 56.049 but...).
I use groff 1.17

-- 
Deux choses remplissent le coeur d'une admiration et d'une vénération
toujours nouvelles et toujours croissantes, à mesure que la réflexion
s'y attache et s'y applique : le ciel étoilé au-dessus de moi et la loi
morale en moi. (Emmanuel Kant)