Re: reading all data from FIFO with 'dd' without blocking

[Bernhard Voelker]

On 11/19/2014 01:40 AM, Assaf Gordon wrote:
> Hello,
>
> I'm looking for a way to read all available data from a FIFO without blocking, 
> using coreutils (or other unix shell commands?).
>
> The use-case:
> 1. A program (inotifywait) waits for a file-system event, then writes several 
> lines to a pipe.
> 2. Another script waits for those events, and runs a program upon changes.
>
> A simple method would be:
> ===
>     SRC=/my/data/directory
>
>     mkfifo myfifo
>     ### InotifyWait will write several lines to "myfifo" on every file change 
> in '$SRC'
>     inotifywait --outfile myfifo --monitor "$SRC" &
>
>     ### Wait for events, and run a program after each file change
>     while read A ; do
>        ## call external program
>        my-program
>     done < myfifo
> ===
>
> I want to minimize the amount of times I run 'my-program'.
> That is: if 'inotifywait' reported 10 changed files in ~20 lines to 'myfifo' 
> (in one quick burst),
> I want to group them together.
>
> A better solution, using 'timeout' and 'cat' is:
> ===
>     SRC=/my/data/directory
>
>     mkfifo myfifo
>     ### InotifyWait will write several lines to "myfifo" on every file change 
> in '$SRC'
>     inotifywait --outfile myfifo --monitor "$SRC" &
>
>     ### Wait for events, and run a program after each file change
>     while read A ; do
>        ## Consume any events that happen in the next 1 second
>        timeout 1 cat > /dev/null
>
>        ## call external program
>        my-program
>     done < myfifo
> ===

I'm still not sure what exactly you want to avoid, but what about
something around these lines:

  while timeout 10 head -n 20 ; do
    my-program
  done < myfifo

This ensures that 'my-program' is run every 10s while running more
often when 20 lines have appeared in the fifo.

But this still somehow contradicts to the idea of inotify ...
can't you read the inotify output from within 'my-program'?

Have a nice day,
Berny