[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: gnulib/gnulib-tool: line 1512: echo: write error: Broken pipe
From: |
Bruno Haible |
Subject: |
Re: gnulib/gnulib-tool: line 1512: echo: write error: Broken pipe |
Date: |
Sun, 31 Aug 2008 01:39:41 +0200 |
User-agent: |
KMail/1.5.4 |
Eric Blake explained on 2008-06-30:
> echo "$handledmodules" | LC_ALL=C join -v 2 - "$tmp"/queued-modules
>
> > 1) The 'join' command (or 'join --nocheck-order' in coreutils 6.11 or newer)
> > apparently closes its standard input and exits when the contents of the
> > second file (small enough or even empty) allows to determine the complete
> > 'join' output - without having read all of its standard input.
>
> Closing stdin on exit is expected by POSIX. What's more, if stdin is
> seekable, it must be reset back to the next position that was not
> consumed by join. But in this case, stdin is a pipe, so the state of
> the pipe is explicitly called out as unspecified by POSIX, so it is
> neither required to be fully consumed, nor to avoid losing data that
> was buffered but not consumed by join.
>
> >
> > 2) The kernel then transforms the pipe into a pipe with no readers.
>
> Correct - if join finishes prior to echo, this is expected.
>
> >
> > 3) The 'echo' command, when writing to a pipe with no readers, gets a
> > SIGPIPE.
>
> Yes. But it depends on whether the shell is trapping/ignoring SIGPIPE
> before this is detectable as a write error instead of an abrupt exit.
>
> >
> > 4) The 'echo' command or its parent shell prints an error message on stderr.
>
> Yes, echo (and all POSIX utilities) are required to print an error message
> to stderr if exiting normally with non-zero status. And if SIGPIPE is
> being ignored, then echo must exit normally with non-zero status
> (if SIGPIPE is not being ignored, then the exit is due to a signal, so
> nothing need be printed).
>
> >
> > Are these 4 behaviours all correct (POSIX compliant and to be expected)?
> > If not, which one is incorrect/buggy?
>
> It seems like they are all expected, so you should either change
> gnulib-tool to not ignore SIGPIPE before using echo, or you should
> change it to silence stderr when it is known that a piped stdout
> might be closed early.
Thank you for explaining all these. If 'join' was to be blamed for closing
stdin too early, I would have applied the attached patch "blaming-join".
But more generally: How should programs react on SIGPIPE? I think the
answer is: If the program produces only output to stdout, then terminating
without an error message is the right behaviour. But if the program has
side effects other than writing to stdout, then the program should show an
error message.
The POSIX default behaviour is to terminate without an error message; this
is the right thing for the classical Unix filters which read from stdin and
write to stdout. For gnulib-tool as a whole, it is not right; that's the reason
why we have the 'trap' command in gnulib-tool line 1100.
In a pipe like
{ while true; do echo foo; done; } | head -10
strace shows me that "head -10" is closing stdin early.
With bash-3.2.33:
{ while true; do echo foo; done; } | head -10 => no error message, stops
{ while true; do /bin/echo foo; done; } | head -10 => no error message, hangs
trap '' SIGPIPE
{ while true; do echo foo; done; } | head -10 => error message, loops
{ while true; do /bin/echo foo; done; } | head -10 => error message, hangs
So it seems that something in Sam's environment is setting the SIGPIPE default
behaviour to the empty command. The output of Sam's test commands in
<http://lists.gnu.org/archive/html/bug-gnulib/2008-06/msg00296.html>
(where he had no error message) indicates that this SIGPIPE default comes in
while executing gnulib-tool somewhere.
I'm applying this. Sam, please speak up if the problem persists.
2008-08-30 Bruno Haible <address@hidden>
* gnulib-tool (func_reset_sigpipe): New function.
(func_get_automake_snippet, func_modules_transitive_closure,
func_import): Invoke it before a join command that reads from stdin,
to avoid "echo: write error: Broken pipe" error messages on stderr.
Reported by Sam Steingold <address@hidden>.
--- gnulib-tool.orig 2008-08-31 01:36:52.000000000 +0200
+++ gnulib-tool 2008-08-31 01:36:46.000000000 +0200
@@ -548,6 +548,31 @@
fi
}
+# func_reset_sigpipe
+# Resets SIGPIPE to its default behaviour. SIGPIPE is signalled when a process
+# writes into a pipe with no readers, i.e. a pipe where all readers have
+# already closed their file descriptor that read from it or exited entirely.
+# The default behaviour is to terminate the current process without an error
+# message.
+# When "trap '' SIGPIPE" is in effect, the behaviour (at least with bash) is to
+# terminate the current process with an error message.
+# This function should be called at the beginning of a command that only
+# produces output to stdout (i.e. no side effects!), when the command that
+# will read from this pipe might prematurely exit or close its standard input
+# descriptor.
+if test -n "$BASH_VERSION"; then
+ # The problem has only been reported with bash.
+ # Note that Solaris sh does not understand "trap - SIGPIPE".
+ func_reset_sigpipe ()
+ {
+ trap - SIGPIPE
+ }
+else
+ func_reset_sigpipe ()
+ {
+ }
+fi
+
# Ensure an 'echo' command that does not interpret backslashes.
# Test cases:
# echo '\n' | wc -l prints 1 when OK, 2 when KO
@@ -1342,7 +1367,8 @@
done | sed -e 's,^lib/,,'`
# Remove $already_mentioned_files from $lib_files.
echo "$lib_files" | LC_ALL=C sort -u > "$tmp"/lib-files
- extra_files=`for f in $already_mentioned_files; do echo $f; done \
+ extra_files=`func_reset_sigpipe; \
+ for f in $already_mentioned_files; do echo $f; done \
| LC_ALL=C sort -u | LC_ALL=C join -v 2 - "$tmp"/lib-files`
if test -n "$extra_files"; then
echo "EXTRA_DIST +=" $extra_files
@@ -1509,7 +1535,7 @@
handledmodules=`for m in $handledmodules $inmodules_this_round; do echo
$m; done | LC_ALL=C sort -u`
# Remove $handledmodules from $inmodules.
for m in $inmodules; do echo $m; done | LC_ALL=C sort -u >
"$tmp"/queued-modules
- inmodules=`echo "$handledmodules" | LC_ALL=C join -v 2 -
"$tmp"/queued-modules`
+ inmodules=`func_reset_sigpipe; echo "$handledmodules" | LC_ALL=C join -v 2
- "$tmp"/queued-modules`
done
modules=`for m in $outmodules; do echo $m; done | LC_ALL=C sort -u`
rm -f "$tmp"/queued-modules
@@ -2406,7 +2432,7 @@
fi
# Determine tests-related module list.
echo "$final_modules" | LC_ALL=C sort -u > "$tmp"/final-modules
- testsrelated_modules=`echo "$main_modules" | LC_ALL=C sort -u | LC_ALL=C
join -v 2 - "$tmp"/final-modules`
+ testsrelated_modules=`func_reset_sigpipe; echo "$main_modules" | LC_ALL=C
sort -u | LC_ALL=C join -v 2 - "$tmp"/final-modules`
if test $verbose -ge 1; then
echo "Tests-related module list:"
echo "$testsrelated_modules" | sed -e 's/^/ /'
@@ -3272,10 +3298,12 @@
if test -f "$destdir/$dir$ignore"; then
if test -n "$dir_added" || test -n "$dir_removed"; then
sed -e "s|^$anchor||" < "$destdir/$dir$ignore" | LC_ALL=C sort >
"$tmp"/ignore
- echo "$dir_added" | sed -e '/^$/d' | LC_ALL=C sort -u \
- | LC_ALL=C join -v 2 "$tmp"/ignore - > "$tmp"/ignore-added
- echo "$dir_removed" | sed -e '/^$/d' | LC_ALL=C sort -u \
- | LC_ALL=C join -v 2 "$tmp"/ignore - > "$tmp"/ignore-removed
+ (func_reset_sigpipe
+ echo "$dir_added" | sed -e '/^$/d' | LC_ALL=C sort -u \
+ | LC_ALL=C join -v 2 "$tmp"/ignore - > "$tmp"/ignore-added
+ echo "$dir_removed" | sed -e '/^$/d' | LC_ALL=C sort -u \
+ | LC_ALL=C join -v 2 "$tmp"/ignore - > "$tmp"/ignore-removed
+ )
if test -s "$tmp"/ignore-added || test -s "$tmp"/ignore-removed;
then
if $doit; then
echo "Updating $destdir/$dir$ignore (backup in
$destdir/$dir${ignore}~)"
blaming-join
Description: Text Data
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Re: gnulib/gnulib-tool: line 1512: echo: write error: Broken pipe,
Bruno Haible <=