bug-bash
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: using mapfile is extreamly slow compared to oldfashinod ways to read


From: Chris F.A. Johnson
Subject: Re: using mapfile is extreamly slow compared to oldfashinod ways to read files
Date: Thu, 26 Mar 2009 17:59:14 -0400 (EDT)
User-agent: Alpine 1.00 (LRH 882 2007-12-20)

On Thu, 26 Mar 2009, Lennart Schultz wrote:

I have a bash script which reads about 250000 lines of xml code generating
about 850 files with information extracted from the xml file.
It uses the construct:

while read line
do
  case "$line" in
  ....
done < file

and this takes a little less than 2 minutes

Trying to use mapfile I changed the above construct to:

mapfile  < file
for i in "address@hidden"
do
  line=$(echo $i) # strip leading blanks
  case "$line" in
  ....
done

With this change the job now takes more than 48 minutes. :(

   As has already been suggested, the time it almost certainly taken
   up in the command substitution which you perform on every line.

   If you want to remove leading spaces, it would be better to use a
   single command to do that before reading with mapfile, e,g,:

mapfile < <(sed 's/^ *//' file)

   If you want to remove trailing spaces as well:

mapfile < <(sed -e 's/^ *//' -e 's/ *$//' file)

   Chet, how about an option to mapfile that strips leading and/or
   trailing spaces?

   Another useful option would be to remove newlines.

--
   Chris F.A. Johnson, webmaster         <http://woodbine-gerrard.com>
   ========= Do not reply to the From: address; use Reply-To: ========
   Author:
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)




reply via email to

[Prev in Thread] Current Thread [Next in Thread]