RE: New Feature Desired for tail

[Brian McQueen]

I'm glad to hear about that program but it does not meet the need I am 
describing.  Even tail will work on several files, but that's not what I am 
interested in.  The most important featur is responding when new lines are 
added to a log file.  Simply watching it not enough.  When new lines are added 
to the log file, my alert must be triggered, and I want those new lines to be 
sent into my alert via the stdio model.  

The current tail does send things on, but it goes on forever:

tail -f files | my_alert

Tail will keep on running and pumping lines into my_alert.

If tail were to find a few new lines and send them off and die, then my_alert 
is a very simple script to write - this is what I want. 

I want to know if ANY lines came in within the last x seconds.  The example 
above is to check for errors every 2 seconds and trigger my alarm if any have 
occurred.  But that may be more than is needed, maybe once per day is enough 
for some circumstances.  If I have to write this into the alert, then its much 
more complicated then it needs to be, but if this timer is put into tail then 
its all simple Unix filter IO.

Imagine there are a series of 30 minute tests you want to run on a server.  
Then you set up this tail on the log file so after each test case, the log file 
lines are sent off to make a report for each test.  Also if the program is 
signaled, then it stops immediately, sending all lines to stdout and terminates 
immediately.  So when a test finishes early, instead of waiting for the full 30 
minute time interval, its Ctrl-C and the captured lines are sent to stdout and 
captured in the test report.



-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of James Youngman
Sent: Thursday, May 07, 2009 3:59 AM
To: Brian McQueen
Cc: address@hidden
Subject: Re: New Feature Desired for tail

On Wed, May 6, 2009 at 11:15 PM, Brian McQueen <address@hidden> wrote:
> The new feature is demonstrated by a wrapper script around tail which
> gives me the ability to use tail to drive arbitrary alerts like this
> (only the core concept lines are shown):
>
>
>
> # put it into the background
>
> tail -n 0 -f error_file > working_file &
>
>
>
> #wait for some lines to arrive
>
> while ! test -s working_file
>
> do
>
>
>
>    echo nothing yet...  Going to sleep for:timeout:$timeout: >&2
>
>    sleep $timeout
>
>
>
> done
>
>
>
> #got some lines so cat them
>
> cat working_file
>
>
>
> This allows me to watch a file for maybe 10 second intervals, and grab
> all lines that arrived during that time interval.  If nothing arrived,
> then it keeps on waiting.  This effectively allows me to drive shell
> scripts with stdin model, but only on when new lines arrive.
>
>
>
> It is used like this:
>
>
>
> ./recent_line_tail error_log | alarm_handler_script
>
>
>
> This functionality could be put into tail with a single new option.
> Proposed Usage (a for alert, or maybe n for new?):
>
>
>
> tail -a 23 error_log
>
>
>
> I propose this would check for output each 23 seconds, and if it finds
> any it will cat it and stop.  If there are no lines then it would wait
> another 23 seconds.

Why not use something designed for the purpose of watching log files,
for example http://www.fourmilab.ch/webtools/logtail/ but ... ?

James.