[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
'date' enhancement: flush in format string
From: |
Darko Veberic |
Subject: |
'date' enhancement: flush in format string |
Date: |
Thu, 11 Aug 2011 23:54:08 +0200 |
User-agent: |
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110617 Thunderbird/3.1.11 |
dear developers,
with awk i have been processing large text files where the first field
was a human-readable date that had to be converted to unix second. for
this purpose i was using an awk pipe call
{d="date -u -d "$1" +%s"; d | getline t; close(d); print t,...}
where $1 was holding the date string. unfortunately the files contained
on the order of million such lines and awk was able to process only
~2000 lines per second. the slowness was mostly due to the spawning of
the "date" process for each line.
then it occurred to me that i could use "date" as a co-process in awk
directly, spawning "date" only once and using the date's -f redirected
to standard input:
BEGIN{d="date -u -f- +%s"}
{print $1 |& d; d |& getline t; print t,...}
END{close(d)}
unfortunately "date" is not flushing its output after each input line is
processed, resulting in awk's getline deadlock.
enhancement request:
is it possible to add another % symbol to the date output format, say %f
(not taken yet) which would trigger output flush, ie i would modify my
example above into
date -u -f- +%s%f
this would really help in my case!
best regards,
%^>
--
darko.
asst.prof.dr. Darko Veberic --------------
Laboratory for Astroparticle Physics
University of Nova Gorica
Vipavska 13, SI-5000 Nova Gorica, Slovenia
mobile: +386 41 860 861
.phone: +386 5 3315 255
...fax: +386 5 3315 385
------------------------------------------
Department for Theoretical Physics
Jozef Stefan Institute,
Jamova 39, SI-1000 Ljubljana, Slovenia
------------------------------------------
CERN, NA61/SHINE, 892 R-A16 Z02200
CH-1211 Geneve 23, Switzerland
phone: +41 22 76 764 89
------------------------------------------
- 'date' enhancement: flush in format string,
Darko Veberic <=