[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Proposed new M4 feature -- dependency rule creation
From: |
David Warme |
Subject: |
Proposed new M4 feature -- dependency rule creation |
Date: |
Mon, 29 Nov 2010 14:40:17 -0500 (EST) |
Hi M4 fans.
Many of us have long wished that M4 would automatically generate
Makefile dependencies. This missing feature has arguably been on the
TODO list for 15 years or more.
Earlier this year I submitted a patch for it. More recently,
Lorenzo Di Gregorio independently submitted a different patch. So the
two of us went off-list to merge our work. In the process, we
discussed some very wide-ranging practical experience regarding
automatic generation of dependency rules, and different ways of
integrating such processes into Makefiles.
Ultimately, we decided to "reboot" -- in other words, to abandon both
our specific patches, and to start over with a purely functional
design. That process went like this:
1. Start with a "mature" implementation of "makedep" functionality
(we used GCC version 4.4.5).
2. Make all necessary adjustments appropriate to the context of M4.
3. Distill it all down into a concrete, written functional
specification.
Lorenzo and I had quite a bit of back and forth iteration during steps
2 and 3. In order to spare the M4 developer community much of the
same effort that Lorenzo and I went through, we present below brief
rationale for each of the design decisions we made while arriving at
the final functional specification, which we present last.
The purpose of this post is to solicit further input from the M4
developers to identify any "blind spots" that Lorenzo and I might have
overlooked. Hopefully we can avoid rehashing decisions already made,
unless they involve a significant error or omission on our part.
Thank you, and sincerely,
David Warme
A brief summary of GCC's existing "makedep" functionality is as
follows:
========================================================================
gcc-4.4.5 provides the following "makedep" type features:
-M Output dependency *instead* of compiling.
-MD Generate dependency AND compile file in one run.
-MM Omit system include files (and files included only indirectly
from such system include files).
-MMD Like -MD, but omits system includes like -MM.
-MF path
Write dependency rules to given path instead of stdout.
-MG Assume missing headers are generated files and add them as
dependencies anyway.
-MP Add a phony target for each dependency other than the main
file (i.e., every file that is the subject of an #include
directive).
-MT target
Specify the dependency rule target verbatim. Multiple "-MT
target" options are OK, and are concatenated with separating
spaces.
-MQ target
Like -MT, but quotes characters that make handles specially.
========================================================================
We then identified the following simplifications / modifications that
are appropriate in the context of GNU m4:
1. GNU m4 has no well-established body of "system" m4 include files.
There is therefore no reason to provide -MM or -MMD capability.
2. In m4, the macro expanded output always goes to stdout. In order to
prevent mingling of macro expanded output with dependency rule
output, the generated dependency rules are always written to a
separate file specified analogously to "gcc -MF file.dep".
3. GNU m4 writes all macro expanded output to stdout, so m4 never really
knows the name of the target file being produced. (Consider, e.g.,
when stdout is the write end of a pipe.) The user must therefore
always explicitly specify the target to use in the generated rule, in
a manner analogous to "gcc -MT target".
4. The ability to specify "-MT target" multiple times is unnecessary.
The same effect can be obtained using -MT once with a target string
of, e.g., "target1 target2 target3".
5. Given "-MT target", there is no pressing need for "-MQ target":
a. Such quoting would cater to the details of one specific "make"
implementation (potentially at the expense of all other makes).
b. The user can perform this quoting operation externally to m4 and
pass the resulting target string in analogously to
-MT "$requoted_target_string".
6. In m4, there is a need to provide two versions of the -MG flag:
a. One that affects include(file_name), and
b. One that affects sinclude(file_name).
Use of either flag makes the macro expanded output incorrect, since
the missing file is replaced with an empty file during that run of
m4. To completely round out this functionality, we also provide:
c. An option that affects files listed on the command line, and
d. An option that means "all" (a, b, and c).
Any missing file that becomes a dependency under any of these
options appears in the generated dependency exactly as given to the
include() or sinclude() macros, or on the command line, respectively;
no prefixes from any -Isearch_dir options are applied. (Consider the
alternative: if one or more -I options are specified, how would M4
know which (if any) prefix to use for a file that is not found in
any of these places?) These options assume, of course, that the
missing files will ultimately NOT include() or sinclude() any
additional files -- if they do, then these additional files will be
missing from the generated dependency rule.
7. In GCC, there can be a huge difference in CPU time between -M mode
(generate dependencies only -- only the preprocessor is run) versus
-MD mode (generate dependencies AND perform compilation -- where all
preprocessing, compilation, optimization and assembly are performed).
This performance difference is lacking in m4, since the entire macro
expansion process must still be performed in either case. So long as
the user has the ability to independently generate/keep or discard
both output streams (the macro expanded output and the dependency
rule output), then there is no need for distinct -M versus -MD modes
in m4. A single mode retains the fundamental capability to run m4 in
a manner that generates: (a) macro output only, (b) dependencies only,
(c) macro output together with dependencies, or (d) neither.
8. As a consequence of items 2, 3 and 7, invoking m4's "makedep"
feature is therefore always analogous to
gcc -MD -MF dependency_file -MT target
where dependency_file and target must always be explicitly
specified.
9. We cannot replicate the GCC option syntax in m4 because -Manything is
already allocated to existing m4 features. The present design uses
options that all match the pattern "--makedep*". Some of the
resulting option names are very long. Note that users typically
will not be typing these options very often -- they will most often
be issued either (a) from Makefile "boilerplate" (e.g., pattern
rules), or (b) by other tools such as automake. There are no
single-character versions of these options -- such short forms should
be reserved for options that users would type more frequently.
10. The -MP option of GCC actually causes a "phony" target dummy rule
to be created for each file that is the subject of at least one
#include directive. Files mentioned on the GCC command line do
not get a dummy rule (unless they are also #include'd from
somewhere). These semantics are somewhat subtle and inflexible.
We address this in m4 by providing separate options controlling
dummy rule generation for include(), sinclude() and files listed
on the command line. There is also a single "all" option that
enables dummy rule generation for all three classes at once.
These rationale have given rise to the following proposed user interface
for the GNU m4 "makedep" feature. It is phrased similarly to what we
would expect to appear in the documentation of these options in the
updated GNU m4 manual.
========================================================================
Proposed Functional Specification
---------------------------------
Options controlling generation of Makefile dependency rules:
Makefile dependency rules can be automatically generated by specifying
both the --makedep=PATHNAME and --makedep-target=TARGET options.
--makedep=PATHNAME causes m4 to generate a dependency rule into the file
specified by PATHNAME. Macro expansion output is still written to
stdout as normal. This option is analogous to the -MF option of GCC.
--makedep-target=TARGET specifies TARGET to be the target of the
generated dependency rule. The string TARGET is used verbatim. The
string TARGET can contain several logical targets separated by spaces.
It is the user's responsibility to properly express characters that
make handles specially (such as '$', or spaces within file names).
Since m4 sends its macro expansion output to stdout, it never really
knows the name of the target file being generated, so the target must
always be specified explicitly by the user with this option. This
option is analogous to the -MT option of GCC (except that GCC allows
-MT to be specified multiple times).
Note that the --makedep=PATHNAME and --makedep-target=TARGET options
must either (a) both be specified, or (b) neither be specified. They
cannot be used independently of each other.
The following additional options can also be used when dependency rules
are being generated (i.e., these options are only valid when both
--makedep=PATHNAME and --makedep=TARGET have also been specified):
--makedep-gen-missing-include causes m4 to assume that any file
included via the include() macro that is missing (i.e., does not
exist) is an automatically generated include file. M4 includes such
missing files as dependencies in the generated rule regardless. In
this case the dependency appears exactly as specified in the argument
to include() and is not modified by any -Isearch_dir prefixes. Note
that the macro expansion output generated to stdout will be incorrect
when this happens because the missing include file is assumed to be an
empty file. This option causes the m4 include() macro to behave like
sinclude(), except that a warning message is produced on stderr to
indicate that the requested file was missing. This option is
analogous to the -MG option of GCC.
--makedep-gen-missing-sinclude causes m4 to assume that any file
included via the sinclude() macro that is missing (i.e., does not
exist) is an automatically generated include file. M4 includes such
missing files as dependencies in the generated rule regardless. In
this case the dependency appears exactly as specified in the argument
to sinclude() and is not modified by any -Isearch_dir prefixes. Note
that the macro expansion output generated to stdout will be incorrect
when this happens because the missing include file is assumed to be an
empty file. This option does not alter sinclude()'s behavior of
silently ignoring requests to sinclude() files that do not exist.
--makedep-gen-missing-argfiles causes m4 to assume that any file listed
on the command line that is missing (i.e., does not exist) is an
automatically generated file. M4 includes such missing files as
dependencies in the generated rule regardless. In this case the
dependency appears exactly as specified on the command line and is not
modified by any -Isearch_dir prefixes. Note that the macro expansion
output generated to stdout will be incorrect when this happens because
the missing file is assumed to be an empty file. A warning is
produced on stderr for each missing command line file handled in
this manner.
--makedep-gen-missing-all is equivalent to specifying all three options:
--makedep-gen-missing-include, --makedep-gen-missing-sinclude, and
--makedep-gen-missing-argfiles.
Note that the above --makedep-gen-missing-* options assume that the
missing files will ultimately NOT include() or sinclude() any additional
files -- if they do, then these additional files will be missing from
the generated dependency rules.
The following options control the generation of "phony" targets for
certain classes of dependencies. These dummy rules are used to work
around errors make gives if you remove files without updating the
Makefile to match. Dependencies that match one or more of these classes
cause a single dummy rule to be generated for them:
--makedep-phony-include causes m4 to generate a "phony" target for each
file that is the subject of an include() macro. This option is
analogous to the -MP option of GCC.
--makedep-phony-sinclude causes m4 to generate a "phony" target for each
file that is the subject of an sinclude() macro.
--makedep-phony-argfiles causes m4 to generate a "phony" target for each
file that is specified on the command line.
--makedep-phony-all is equivalent to specifying all three options:
--makedep-phony-include --makedep-phony-sinclude --makedep-phony-argfiles.
========================================================================
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Proposed new M4 feature -- dependency rule creation,
David Warme <=