Re: Does head util cause SIGPIPE?

[Bernhard Voelker]

[adding the findutils ML]

On 2019-10-25 09:56, Ray Satiro wrote:
> Recently I tracked down a delay in some scripts to this line:
> 
> find / -name filename* 2>/dev/null | head -n 1
> 
> It appeared that the searching continued after the first result was 
> found. I expected head would terminate and SIGPIPE would be sent which 
> would cause find to terminate, but that did not happen. Since I was in 
> cygwin I thought maybe it was a Windows issue but I tried in Ubuntu 16 
> with an aribtrary file and noticed the same thing. I also tried 
> disabling any pipe trap and monitoring with strace. It looks like head 
> exits but find continues searching. This was only observable the first 
> run since some caching seems to be done which makes subsequent runs 
> complete too fast to tell, but if I reset the VM I can reproduce.
> 
> owner@ubuntu1604-x64-vm:~$ ( trap '' pipe; find / -name initrd* 
> 2>/dev/null | strace -e 'trace=!all' head -n 1)
> /initrd.img
> +++ exited with 0 +++
> (few seconds wait)

'head' just terminates after it has read the given number of lines.

> Since we only need the first line I can just use find options -print 
> -quit and skip piping to head. But say we needed the first n results, 
> how would I do that with head and get find to terminate rather than 
> continue searching?
> 
> find (GNU findutils) 4.7.0-git
> head (GNU coreutils) 8.25

... and 'find' properly terminates after receiving EPIPE:

  $ strace -ve write find /dev | head -n 1 >/dev/null
  write(1, "/dev\n/dev/.blkid.tab\n/dev/.blkid"..., 4096) = 4096
  write(1, "chi_HDS721010CLA330_JP2911N03AR0"..., 4096) = -1 EPIPE (Broken pipe)
  --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=21014, si_uid=1000} ---
  +++ killed by SIGPIPE +++

As you can see from above strace output, find does some buffering of the
output, i.e., it does not write one line per entry.
Thus, I assume that searching through the files until the first 4k output
buffer is full takes a while on your system.

To confirm, please try with turning this buffering off with 'stdbuf':

  $ trace -vfe write stdbuf -o 0 find /dev | head -n 1 >/dev/null
  write(1, "/dev\n", 5)                   = 5
  write(1, "/dev/.blkid.tab\n", 16)       = -1 EPIPE (Broken pipe)
  --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=21098, si_uid=1000} ---
  +++ killed by SIGPIPE +++

With more depth, another kind of buffering comes into play:
'find' uses gnulib's FTS module to read the next directory hierarchy.
This means, that although 'head' might terminate at e.g. 10 items,
'find' already has read the entries for a couple of entries more.

Finally, the situation becomes more complicated if it is not 'find' which
is writing into the broken pipe, but a -exec'ed tool: in that case,
that tool terminates due to EPIPE but find will continue processing
with the next entry:

  $ mkdir x && cd x
  $ seq 10 | xargs touch
  $ find . -exec echo '{}' \; | head -n 3
  .
  ./10
  ./9
  find: 'echo' terminated by signal 13
  find: 'echo' terminated by signal 13
  find: 'echo' terminated by signal 13
  find: 'echo' terminated by signal 13
  find: 'echo' terminated by signal 13
  find: 'echo' terminated by signal 13
  find: 'echo' terminated by signal 13
  find: 'echo' terminated by signal 13

FWIW: To work around the above, one would use the false value of -exec in the
case the command to be run failed:

  $ find . -exec echo '{}' \; -o -quit | head -n 3
  .
  ./10
  ./9
  find: 'echo' terminated by signal 13

Does this describe the effects you see?

Have a nice day,
Berny