[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: frozen files format 2

From: Gary V. Vaughan
Subject: Re: frozen files format 2
Date: Thu, 22 Jun 2006 15:44:25 +0100
User-agent: Thunderbird (X11/20060519)

Hi Eric!

Eric Blake wrote:
> In the "sysval and doc fixes" thread:
>>> But it does beg the question - should frozen files preserve sysval?
>>> It's not worth changing this for 1.4.x, but we should probably add
>>> another command to frozen file format 2 in m4-2.0 to preserve nonzero
>>> sysval over freezing.
> Another thing that should be preserved over freezing is m4wrap text.
> Right now:
> $ echo "m4wrap(\`hi
> ')dnl" > a.m4
> $ echo "define(\`hi', \`HI')dnl" > b.m4
> $ m4 a.m4 b.m4
> HI
> $ m4 -F a.m4f a.m4
> hi
> $ m4 -R a.m4f b.m4
> $
> Oops - the semantics of a followed by b in the same m4 run are not met,
> since m4wrap was expanded during freezing instead of saved for use after
> end of input on reload.

D'oh!  Another nice catch though.

> I propose that version 2 of frozen files adds the
> "'W' length '\n' string '\n'" command, which adds the string to the end of
> the m4wrap text (assuming we also fix m4wrap to be FIFO).

Agreed.  And I think making m4wrap FIFO is fine as long as it doesn't
break autoconf-2.60, and provided we put a giant warning in README,
and at the end of ./configure output, that using m4-2.x with autoconf
< 2.60 is doomed to failure.

> Back to the topic of sysval, just adding a command to set just sysval is
> not scalable; who knows what other modules will have state to save,
> although we don't want to add a command letter for every module.  Instead,
> I propose extending the M command.  Right now, M only takes a single
> argument, the module name to load.  I propose the alternative "'M' length1
> ',' length2 '\n' string1 string2 '\n'" command, generated on freezing if a
> module has a freeze hook installed, and on reload, feed string2 to the
> reload hook of the module (if a reload hook is present); if a module has a
> reload hook but there is no string2, then call the reload hook with the
> empty string.  Then, the m4 module, which provides the sysval stateful
> macro, would also provide a freeze hook which outputs the value of sysval
> as a string, and a reload hook which parses the string and sets the value
> of sysval.

An excellent idea!!

> This means that m4module.h would need to be expanded to add
> M4FREEZE_HANDLER (called when a module is being frozen, to give the module
> a chance to output a string according to its own formatting conventions of
> state to preserve), and M4RELOAD_HANDLER (called when a module is being
> reloaded, to give the module to restore frozen state).


> My next question concerns text builtins like __unix__.  Right now, the gnu
> module determines at compile-time whether __unix__ or __windows__ should
> be defined.  But a frozen file created on one type of machine, and
> reloaded on another, will result in the wrong macro being defined.  Is
> this okay,

No, this is bad.

> or should we be adding some smarts to the module freezing which
> notices when __unix__ was never redefined by the user on a unix machine,
> omit it from the frozen file; and on reload on a windows machine,
> instantiate __windows__ if neither one of the two were explicitly
> mentioned in the frozen file?

I'm not sure we can do that either.  If part of the frozen state was
arrived at by examining __unix__ et. al., and then we continue execution
on a __windows__ machine things could become unpredictable.  I think the
best we can do is document the potential problems, and after reloading
state, recalculate whether __unix__ et. al. should be defined... perhaps
throwing a warning if this is different to what was in the frozen state?

For scalability we might want to allow marking of macros that will be
recalculated at module load time (and issue warnings if they are
different to a reloaded state), along with corresponding code in
m4_module_load() to implement the checking and warning -- maybe also
splitting the recalculation code into a M4STATICMACRO_HANDLER?

> Next, should we allow partial input?  Right now, m4 chokes if a file is in
> the middle of a macro, even if the next file could complete the story:
> $ echo "len(" > c.m4
> $ echo "abc)" > d.m4
> $ m4 c.m4 d.m4
> NONE:0: m4: ERROR: EOF in argument list
> $ cat c.m4 d.m4 | m4
> 3
> $ m4 -F c.m4f c.m4
> NONE:0: m4: ERROR: EOF in argument list
> $
> But since we also refuse to create c.m4f, our behavior of freezing matches
> our normal behavior.  I'll have to check how Solaris behaves in this case,
> but right now, I don't think I'm inclined to change behavior here.

ACK.  An interesting feature to add post-2.0 though :-)

> Then there is the question of diversion 0.  Right now, the frozen file
> format supports reloading diversion 0, which dumps to stdout; but freezing
> never generates a diversion 0 in the frozen file (instead, m4 already
> printed the text to stdout prior to the creation of the frozen file).  So
> an application that wants to preserve output is permitted to do something
> like:
> $ m4 -F foo.m4f foo.m4 > foo.out
> $ printf 'D0,%d\n' $(wc -c < foo.out) >> foo.m4f
> $ cat foo.out >> foo.m4f
> $ echo >> foo.m4f
> But it might be nice if we added an option to m4, perhaps named
> --freeze-stdout, which is only valid if -F is also in effect, and which
> tells m4 to buffer diversion 0 into the frozen file rather than straight
> to stdout.

Nice idea.

> Finally, I propose tightening the syntax of frozen files.  Right now, the
> "'V' number '\n'" command can occur anywhere, multiple times or not at
> all, but some of the reload logic in CVS head depends on knowing whether
> we are version 1 or 2.  I propose that the frozen file format be changed
> to require that 'V' always be present as the first non-comment command,
> and reject a frozen file as invalid if some other command appears first or
> if V appears more than once.  This should be upwards compatible with
> existing frozen files, since the output generated by m4 -F already meets
> this, and it is not likely that many people have hand-edited frozen files
> to violate this constraint.  Of all the ideas in this email, this is the
> only idea that we should also fold into m4 1.4.5.

Agreed.  I also think that having a combined loader in m4-2.x was a bad idea.
We would be better to have a 'V1' loader common to m4-1.4.x and all future
releases, and a separate 'V2' loader for m4 >= 2.0.  That way, we can be
sure of not breaking the 'V1' loader with future enhancements in 'V2' and

(FTR, my original idea was to (eventually) have the parser distributed
among the appropriate modules so that the loaded modules would dictate the
entities that might be added to a frozen file, and that reading back a frozen
file might load the appropriate modules to handle those entities.  Maybe
we can still consider this for 'V3' one day).

Gary V. Vaughan      ())_.  address@hidden,gnu.org}
Research Scientist   ( '/   http://blog.azazil.net
GNU Hacker           / )=   http://trac.azazil.net/projects/libtool
Technical Author   `(_~)_   http://sources.redhat.com/autobook

Attachment: signature.asc
Description: OpenPGP digital signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]