Re: Target appends to file?

[Brian Vandenberg]

As an alternative you can use bash and make's .ONESHELL to do your
logging.  I believe this will require bash 4.x (or newer), but I could be
mistaken.  Also, understand this will tie your build to bash because these
features are either unavailable in other shells (like the bourne shell) or
use different syntax to accomplish the same thing.

Here's an example:

.RECIPEPREFIX := >
.ONESHELL:
SHELL := bash
.SHELLFLAGS := -e -o pipefail -c

# note: the leading '@' is to make it not print the recipe before execution
dut:
> @
> exec 1> >(tee -a phase1.log) 2> >(tee -a phase1.log 1>&2)
> echo Phase 1
> ./shell_script
> vlog ${VOPTS}

For my stuff I've added a trap for INT and ERR so it puts a custom message
in the log to help make it easy to find logs for recipes that failed:

dut:
> @
> exec 1> >(tee -a phase1.log) 2> >(tee -a phase1.log 1>&2)
> trap 'result=$$?; trap INT; trap ERR; echo "Exit code: $$result"; exit
$$result' INT ERR
> echo Phase 1
> ./shell_script
> vlog ${VOPTS}


Here's a summary of what's going on in both of the above examples:

The '-e' and '-o pipefail' options make it so /any/ non-zero exit code will
cause my build to fail.  That may not work for everyone but it does for
me.  As an example of how this is useful, we have a recipe that generates
code using a generic template modified using sed before compilation; the
recipe looks something like this:

%.cc : %.whatever
> ${something} | sed -e 'some sed script stuff' | some other thing

With default bash behavior the following will not cause a failure in gnu
make:

blah:
> false | false | true

... because the only exit code that the shell reports back to make is the
last process in the pipeline.  The same thing happens here for slightly
different reasons:

blah:
> false; false; true

... in this case because the only exit code reported back to make is the
last process to finish executing.

The '-e' flag would cause the example with "false; false; true" to fail; it
would also stop executing after the 1st failure.

The '-o pipefail' option will cause the example with "false | false | true"
to fail.  I'm inclined to suppose that the shell will return the exit code
from the 1st process that fails, but I don't know all the gory details so
that's just supposition on my part.  That's been my experience so far, but
I suppose there could be some corner case or another where 2 failures in
the same pipeline could generate different exit codes due to a race
condition, but if that's possible it isn't an issue in our case -- a
failure is a failure.

Next up: the "exec" lines redirect the shell's stdin/stdout -- in this case
to a subshell (note the two >()s).  If you don't want to see the output
while building you could use this instead:

> exec 1> phase1.log 2>&1

... or if you never want to clobber it and instead rely on the developer to
clobber it themselves:

> exec 1>> phase1.log 2>&1

Finally, when the trap is triggered: it saves the exit code, stops trapping
on INT & ERR, prints the exit code, then exits with the same exit code.  It
is initially set to trap on INT (user hits ^C) and ERR (it's the only trap
I could find associated with the script bailing due to the *-e* and *-o
pipefail* options).

The only problem I've seen with this: the trap only executes if it fails in
the middle of executing the script.  If the last command executed generates
a failure then the trap isn't triggered so the message printing the exit
code isn't printed into the log.  While that sucks, at a minimum make is
still failing so that's good.

Additionally, if you want commands executed to be placed in the log you can
set bash's "x" flag through .SHELLFLAGS, but you need to be careful.  If
you use $(shell) there's two situations where the flag will bite you.
Here's two examples to illustrate:

$ cat <<'EOF' | gmake -f -
SHELL := bash
.SHELLFLAGS := -e -o pipefail *-x* -c
$(warning $(shell echo this is a test))
asdf:
> @echo $(printf $@)
EOF
+ echo this is a test
/var/tmp/GmMnaqMv:3: this is a test
++ printf asdf
+ echo asdf
asdf


It's printing the commands being executed to STDERR.  Furthermore, my use
of $(shell) outside the recipe caused something to be printed to STDERR.
If in your $(shell) command you redirect STDERR expecting to do something
useful with its output you'll get more than you expected.

The 2nd example is almost identical, except I'm adding the *-x* flag after
the entire makefile is parsed so a $(shell) executed outside a recipe won't
be affected; however, as you'll see if $(shell) is used inside a recipe
it's also affected:

$ cat <<'EOF' | gmake -f -
SHELL := bash
.SHELLFLAGS := -e -o pipefail -c
$(warning $(shell echo this is a test))
asdf:
> @echo $(printf $@)
# note: don't append; -c needs to be the last parameter
.SHELLFLAGS := -e -o pipefail *-x* -c
EOF
/var/tmp/GmMnaqMv:3: this is a test
++ printf asdf
+ echo asdf
asdf


To resolve this problem you can put *set -x;* at the front of your recipes
that do logging instead of setting it in *.SHELLFLAGS*; this prevents
*$(shell)* inside recipes from having that flag set.

-brian

On Thu, Apr 14, 2016 at 4:59 AM, Lynch, Thomas <address@hidden>
wrote:

> Greetings,
>
>
>
>                 Having a problem append to a file within a target. I have
> the following:
>
>
>
>                 dut:
>
>                                 @echo "Phase 1"
>
>                                 ./shell_script 2>&1 >phase1.log
>
>                                 vlog $(VOPTS) >>phase1.log 2>&1
>
>
>
>                 It's the "vlog" line I'm have trouble with, I've played
> around where to put the stderr/stdout re-direct,
>
>                 and checks the SHELL variable, but can't seem to get it to
> append to the phase1.log. The shell script
>
>                 works fine. Any ideas what I'm doing wrong here? Thanks for
> any help in advance.
>
>
>
>                 -Tom
>
>
>
>
>
>
> _______________________________________________
> Help-make mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/help-make
>
>