Re: CFLAGS and other flag variables

[Alejandro Colomar]

Hi Kaz,

Sorry for answering so late.

On Wed, Jul 17, 2024 at 03:49:27AM GMT, Kaz Kylheku wrote:
> On 2024-07-17 02:11, Alejandro Colomar wrote:
> > Hi Kaz,
> > 
> > On Tue, Jul 16, 2024 at 05:10:00PM GMT, Kaz Kylheku wrote:
> >> because there is a built-in recipe for making .o files
> > 
> > Except that I do:
> > 
> >     MAKEFLAGS += --no-builtin-rules
> >     MAKEFLAGS += --no-builtin-variables
> >     MAKEFLAGS += --warn-undefined-variables
> > 
> > Sorry for not mentioning that.
> 
> OK; by the way, --no-builtin-variables implies --no-builtin-rules.

Hmm, ok.

> Secondly, where are you doing this? It has no effect on the
> currently executing make. If you have rules which call make
> recursively, these flags will affect those invocations.

That's done within the GNUmakefile.  And it's non-recursive.

make(1) already interprets --no-builtin-rules within the Makefile in
the latest version, but --no-builtin-variables seems to be ignored
(although they've fixed that in master already).

        $ git log -1 e9dd614d73fc625845a0de089edae3a8433e30c1
        commit e9dd614d73fc625845a0de089edae3a8433e30c1
        Author: Dmitry Goncharov <dgoncharov@users.sf.net>
        Date:   Mon Feb 20 15:17:24 2023 -0500

            [SV 63821] Don't set up default suffix rules if makefile sets -r
            
            When built-in rules are disabled by adding -r to MAKEFLAGS in the
            makefile, don't add suffix rules at all so that if suffixes are
            added back via .SUFFIXES, the rules are still not there.
            
            * src/main.c (main): Set default suffix rules after parsing 
makefiles.
            * src/default.c (install_default_suffix_rules): Install a default
            suffix rule only if there is no user defined rule.
            * tests/scripts/features/suffixrules: Add tests.
        $ git describe --contains e9dd614d73fc625845a0de089edae3a8433e30c1
        4.4.1~6


> >> Programs should use CFLAGS, CPPFLAGS, LDFLAGS and LDLIBS if those
> >> are set on the make command line or in the environment.
> > 
> > Hmmmm, for the time being, I won't be able to use them from the
> > environment, because that conflicts with
> > 
> >     MAKEFLAGS += --no-builtin-variables
> 
> This does not make external variables go away. If CFLAGS is
> defined on the environment or passed on the command line,
> it still appears inside make. Just make won't define it
> if it doesn't exist.
> 
> > I'll need the next release of make(1) to be able to use an empty default
> > 
> >     CFLAGS ?=
> 
> What release is that?

4.5 maybe?

        $ git log -1 0880e5c86ab7ba92c3ba0f83bab1a230906ee657
        commit 0880e5c86ab7ba92c3ba0f83bab1a230906ee657
        Author: Dmitry Goncharov <dgoncharov@users.sf.net>
        Date:   Sun Apr 30 09:26:29 2023 -0400

            [SV 64107] Disable builtins immediately on -R or -r
            
            Disable builtin variables and rules immediately, when -R or -r is
            added to MAKEFLAGS inside the makefile.
            
            * src/main.c (disable_builtins): Add new function 
disable_builtins().
            (main): Call disable_builtins().
            (reset_makeflags): Call disable_builtins().
            * tests/scripts/options/dash-r: Add tests.
            * tests/scripts/variables/MAKEFLAGS: Update tests.
        $ git describe --contains 0880e5c86ab7ba92c3ba0f83bab1a230906ee657
        fatal: cannot describe '0880e5c86ab7ba92c3ba0f83bab1a230906ee657'
        $ git describe 0880e5c86ab7ba92c3ba0f83bab1a230906ee657
        4.4.1-34-g0880e5c8


> 
> In make 4.1 from 2014, the above CFLAGS ?= defines CFLAGS if CFLAGS doesn't 
> exist.

Which means it _always_ defines CFLAGS, because it always exists (as a
built-in variable), unless you pass -R in the command line, which nobody
does (I think).

> 
> Given
> 
>    # CFLAGS ?=
>    $(info $(CFLAGS))
> 
> if I run "make -R --warn-undefined-variables", I get a warning on $(CFLAGS).
> If I uncomment the CFLAGS ?= line, it goes away.

Yeah, but I'm assuming a user of my makefiles will not pass -R to
make(1) on the command line.  Why would they?

> >> If you want to use built-in recipes, and your code needs
> >> certain cflags (like for instance C dialect selection), then
> >> you can add that to CFLAGS:
> >>
> >>    CFLAGS += -std=c17 -Wmy-favorite-warning-option
> > 
> > There's a problem with that.  My selection would have preference over
> > the user, since it's specified later in the command line.
> 
> That's not the problem with that. The problem with that is that
> if we have this in the Makefile:
> 
>   CFLAGS += --our-important-flag
> 
> and the user adds CFLAGS += --extra-flag on the make command line,
> that presence on the command line obliterates our append above;
> our important flag is gone.

I didn't know it was possible to use += in the command line.
Interesting.  Although, as you say, it seems a bit weird that it
overwrites += specified in the makefile.  :|

        $ cat GNUmakefile 
        FOO := bar
        FOO += baz
        $(info $(FOO))
        $ make FOO=qwe
        qwe
        make: *** No targets.  Stop.
        $ make FOO+=qwe
        qwe
        make: *** No targets.  Stop.
        $ make FOO=asd FOO+=qwe
        asd qwe
        make: *** No targets.  Stop.

It seems it only serves to compose a command line in several arguments,
which is unnecessary, since you can already do that by preparing a shell
variable first and then passing it once to make(1).

> This is the reason why you really want your own private variables,
> and have to define your own recipes which look like this:
> 
>   $(CC) $(CFLAGS) $(MY_CFLAGS) ...
> 
> rather than trying to append to CFLAGS.

Makes sense.

> Because then the user can freely manipulate CFLAGS
> without obliterating your important flags in
> MY_FLAGS.  (If they want to obliterate those, they have to
> learn about MY_CFLAGS.)

I think I prefer

        $(CC) $(MY_CFLAGS) $(CFLAGS) ...

or actually a shorter one:

        CFLAGS_ := $(MY_CFLAGS) $(CFLAGS)
        $(CC) $(CFLAGS_) ...

so that they can more easily obliterate my important cflags if they
want.  They're not that important.  Here are the flags I use:

        $ grepmk -h CFLAGS_ share/mk/
        CFLAGS_        := $(DEFAULT_CFLAGS) $(CFLAGS)
        $ grepmk -h CFLAGS share/mk/
        CFLAGS         :=
        $ grepmk -h DEFAULT_CFLAGS share/mk/
        DEFAULT_CFLAGS := $(COMMON_CFLAGS)
        DEFAULT_CFLAGS += $(GCC_CFLAGS)
        DEFAULT_CFLAGS += $(CLANG_CFLAGS)
        DEFAULT_CFLAGS += -ffat-lto-objects
        $ grepmk -h COMMON_CFLAGS share/mk/
        COMMON_CFLAGS := \
                -O3 \
                -flto \
                -Wall \
                -Wextra \
                -Werror \
                -Wstrict-prototypes \
                -Wdeclaration-after-statement \
                -Wno-unknown-pragmas \
                -Wwrite-strings
        $ grepmk GCC_CFLAGS share/mk/
        GCC_CFLAGS := -fanalyzer
        $ grepmk CLANG_CFLAGS share/mk/
        CLANG_CFLAGS := \
                -Weverything \
                -Wno-c++98-compat \
                -Wno-empty-translation-unit \
                -Wno-gnu-auto-type \
                -Wno-gnu-statement-expression \
                -Wno-language-extension-token \
                -Wno-unused-macros

which are just good defaults.

> It would be useful if Make itself defined this mechanism in more detail;
> that is to say, if the built-in recipes contained more variables for
> more parties, like
> 
>   $(CC) $(CPPFLAGS) $(OWN_CPPFLAGS) $(EXTRA_CPPFLAGS) \
>         $(CFLAGS) $(OWN_CFLAGS) $(EXTRA_CFLAGS) ...
> 
> The names would be standard and everyone would understand that
> distros typically control the plain variables like CFLAGS,
> that their own Makefile can assert its essential flags in the
> OWN_CFLAGS variable, and the user who is trying to build the
> program through the distro build can use EXTRA_CFLAGS
> to customize something (e.g. temporarily during debugging).

Yeah, having some common convention for naming these variables would be
good.  While changing the built-in rules might be problematic with
existing code, stating some standard conventions in the documentation
and the GNU coding standards would be an improvement.

Ideally, those should be consistent with existing practice.

> >> If CFLAGS exist,
> > 
> > The bad part is that CFLAGS _always_ exists (unless you remove it on the
> > command line with -R, which is kind of nonsensical, because whoever runs
> > make(1) already can override it by specifying a different value
> > explicitly).
> 
> Though it exists since it is one of the predefined variables, it
> has a blank value. So for instance CFLAGS ?= -O2 kicks in, if there
> is no CFLAGS on the command line or in the environment, even though
> CFLAGS exists due to being defined by make.

That's true of CFLAGS, but not for example of ARFLAGS.

        $ cat GNUmakefile 
        $(info $(ARFLAGS))
        $ make
        -rv
        make: *** No targets.  Stop.

I'd like to have a convention that I can use with all variables, and not
just rely on the fact that some are unset.  Otherwise, if I use ?= with
CFLAGS because I can, but use := with ARFLAGS because I can't, a user
might be surprised to learn that when it sets CFLAGS in the environment
I use it, but when it sets ARFLAGS in the environment I completely
ignore it.  I prefer to ignore all of them, for consistency.

Hopefully, in make 4.5 I'll be able to set -R in the makefile and use ?=
with all variables.  I'll need to add some conditional that reports an
error if the version of make(1) is too old, though.

> It's almost as if CFLAGS is not really defined. (Except that if you're
> using warning on undefined variables, CFLAGS doesn't trigger
> anything unless you -R it out.)
> 
> > Thankfully, the next version of make(1) will fix that historical bug.
> > 
> >> assume that the user/distro is using that to
> >> control optimization and possibly other code generation options.
> >> You can disable optimization by adding -O0 after $(CFLAGS).
> > 
> > That would be overriding the user's choice, which I don't think is a
> > good idea.
> 
> Right; I only mean temporarily for debugging. The user themselves
> could be doing that. They have CFLAGS coming out of the distro build
> they are using, and don't want to touch that. So using whatever means
> is available in your Makefile, they may be able to add -O0 after
> the CFLAGS.
>
> E.g. if your custom recipes have $(CFLAGS) followed
> by $(EXTRA_CFLAGS), the user can learn about that and specify
> EXTRA_CFLAGS=-O0.

Distro maintainers complain loudly if I their CFLAGS don't override
mine.  Providing a custom way to do that is allegedly uncomfortable for
them.  :)

> 
> (Incidentally, the author of the program should have a say in whether
> optimization applies. If the program does things that break under
> optimization, it would make sense to ship it such that it builds
> -O0 even if a distro specifies -O2.

In such cases (for example, in some invocations of programs, a flag is
strictly necessary for it to work), I hardcode the variable in the
recipe.

$ grepmk -htr _DISTFILE .
$(_DISTFILE): $(_DISTFILES) $(MK) | $$(@D)/
        $(info  $(INFO_)TAR             $@)
        $(TAR) $(TARFLAGS_) -cf $@ -T /dev/null
        $(DISTFILESCMD) \
        | $(SED) 's,^$(srcdir)/,$(_DISTDIR)/,' \
        | $(SORT) \
        | $(XARGS) $(TAR) $(TARFLAGS_) -rf $@ -C $(srcdir) \
                --transform 's,^$(patsubst /%,%,$(_DISTDIR)),$(DISTNAME),'

-cf and -rf are strictly necessary for this to work, so I kept them out
of TARFLAGS.  (Also, being different for the two invocations, I needed
to specify them out, but I would have done so anyway, probably.)

> I have a program that can't work with link-time optimizations (LTO)
> Some distros are starting to enable them, and so I have to counteract
> that in the Makefile. I understand that the distro people like the
> shiny new way to make things go faster, but we have to let it go
> in that program.)

Makes sense.

Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

signature.asc
Description: PGP signature