emacs-bug-tracker
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Emacs-bug-tracker] bug#7357: closed (csplit: memory exhausted when usin


From: GNU bug Tracking System
Subject: [Emacs-bug-tracker] bug#7357: closed (csplit: memory exhausted when using stdout / pipe instead of a file)
Date: Wed, 10 Nov 2010 16:25:02 +0000

Your message dated Wed, 10 Nov 2010 16:29:35 +0000
with message-id <address@hidden>
and subject line Re: bug#7357: csplit: memory exhausted when using stdout / 
pipe instead of a file
has caused the GNU bug report #7357,
regarding csplit: memory exhausted when using stdout / pipe instead of a file
to be marked as done.

(If you believe you have received this mail in error, please contact
address@hidden)


-- 
7357: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=7357
GNU Bug Tracking System
Contact address@hidden with problems
--- Begin Message --- Subject: csplit: memory exhausted when using stdout / pipe instead of a file Date: Tue, 9 Nov 2010 12:02:55 +0100

Hi,

 

When I try to use csplit via stdout from zcat I get the message ‘csplit: memory exhausted’. The .gz file is a mysql dump from its databases (one is particularly large):

 

mysqldump --opt --add-locks --add-drop-table -a –user=foo -p --all-databases | gzip > ./mysql/backup.gz

 

When I unzip the file and then use csplit it works perfectly. However, these files get rather big (8GB+) so I want to let them go through a pipe. When I let the data go through a pipe it fails:

 

address@hidden:~/scripts/csplittest/20101109/mysql$ zcat backup.gz | csplit -f db_ -n 4 - /--\ Current\ Database/ {*}

753

74459

75736

306159

121390

8215

10744920

4984

56142

6113058

788008

67549

513413

216776062

67525

csplit: memory exhausted

2081734963

address@hidden:~/scripts/csplittest/20101109/mysql$ ls -ltr

total 836324

-rw-r--r-- 1 davidh davidh 855553350 2010-11-09 01:35 backup.gz

address@hidden:~/scripts/csplittest/20101109/mysql$ zcat backup.gz > /dev/null

address@hidden:~/scripts/csplittest/20101109/mysql$

 

Environments:

Ubuntu 8.04; Linux xinker 2.6.27-17-generic #1 SMP Fri Mar 12 03:09:00 UTC 2010 i686 GNU/Linux:

address@hidden:~/scripts/csplittest$ aptitude show coreutils

Package: coreutils

Version: 6.10-6ubuntu1

 

Debian Lenny: Linux test 2.6.26-1-686 #1 SMP Fri Mar 13 18:08:45 UTC 2009 i686 GNU/Linux;

test:~# aptitude show coreutils

Package: coreutils

Version: 6.10-6

 

To find the problem I wanted to create a core dump but... I can’t seem to be able to create a core dump (ulimit –c == unlimited) !? Am I missing something simple here or is this a bug? Is zcat too hungry?

 

I would expect the failing command to be very memory friendly so I’m a bit puzzled. Thanks in advance,

 

Met vriendelijke groet,

David Hofstee
Software Engineer Blinker B.V.          


--- End Message ---
--- Begin Message --- Subject: Re: bug#7357: csplit: memory exhausted when using stdout / pipe instead of a file Date: Wed, 10 Nov 2010 16:29:35 +0000 User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
I'll apply this patch in a few hours,
and it will be in the very soon to be released
coreutils 8.7

cheers,
Pádraig.

Attachment: csplit-mem-leak.diff
Description: Text Data


--- End Message ---

reply via email to

[Prev in Thread] Current Thread [Next in Thread]