[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug #34020] RFE: Improve handling of sparse files
From: |
Paolo Bonzini |
Subject: |
[bug #34020] RFE: Improve handling of sparse files |
Date: |
Wed, 21 Dec 2011 15:57:46 +0000 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 |
Follow-up Comment #1, bug #34020 (project grep):
This is not a bug. The problem is not the sparseness of the file. The
problem is that grep works on lines, and lines are separated by LF characters.
So, a file that has 2GB of NULLs still has a single line.
This bug in particular is not fixable when grep's input is a pipe (so you
cannot rewind the input when printing).
grep used to have some old code that used mmap to read the file. We ripped
that out because in normal cases it actually had worse performance than
read(), but in this case it had an advantage: the kernel has the possibility
to swap out (and back in) to the sparse file. It would still require 100% CPU
and can fail on 32-bit machines due to the small address space.
However, care must be taken for code like this:
echo foo > bar
(grep foo; cat) < bar
Here, the "cat" command must output nothing. IIUC the old mmap code did not
advance the file pointer, so that the "cat" would print again the whole
contents of the input file.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?34020>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
- [bug #34020] RFE: Improve handling of sparse files,
Paolo Bonzini <=