[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

'date' enhancement: flush in format string

From: Darko Veberic
Subject: 'date' enhancement: flush in format string
Date: Thu, 11 Aug 2011 23:54:08 +0200
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20110617 Thunderbird/3.1.11

dear developers,

with awk i have been processing large text files where the first field was a human-readable date that had to be converted to unix second. for this purpose i was using an awk pipe call

{d="date -u -d "$1" +%s"; d | getline t; close(d); print t,...}

where $1 was holding the date string. unfortunately the files contained on the order of million such lines and awk was able to process only ~2000 lines per second. the slowness was mostly due to the spawning of the "date" process for each line.

then it occurred to me that i could use "date" as a co-process in awk directly, spawning "date" only once and using the date's -f redirected to standard input:

BEGIN{d="date -u -f- +%s"}
{print $1 |& d; d |& getline t; print t,...}

unfortunately "date" is not flushing its output after each input line is processed, resulting in awk's getline deadlock.

enhancement request:
is it possible to add another % symbol to the date output format, say %f (not taken yet) which would trigger output flush, ie i would modify my example above into

date -u -f- +%s%f

this would really help in my case!

best regards,
darko. Darko Veberic --------------
Laboratory for Astroparticle Physics
University of Nova Gorica
Vipavska 13, SI-5000 Nova Gorica, Slovenia
mobile: +386 41 860 861
.phone: +386 5 3315 255
...fax: +386 5 3315 385
Department for Theoretical Physics
Jozef Stefan Institute,
Jamova 39, SI-1000 Ljubljana, Slovenia
CERN, NA61/SHINE, 892 R-A16 Z02200
CH-1211 Geneve 23, Switzerland
phone: +41 22 76 764 89

reply via email to

[Prev in Thread] Current Thread [Next in Thread]