[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug-wget] Memory Leakage
From: |
thomas hood |
Subject: |
[Bug-wget] Memory Leakage |
Date: |
Mon, 14 Mar 2016 10:59:25 +0000 (UTC) |
Hey,
Using wget recently to handle large amounts of data. Finding it frequently eats
into memory quite a bit.Mar 12 06:32:08 ubuntu kernel: [1439386.493486] Out of
memory: Kill process 63125 (wget) score 661 or sacrifice childMar 12 06:32:08
ubuntu kernel: [1439386.494595] Killed process 63125 (wget) total-vm:4206452kB,
anon-rss:3161380kB, file-rss:0kBThese are the parameters I'm using;
wget -c -nd -b -r -e robots=off --header="Accept: text/html"
--user-agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101
Firefox/40.1" http://$URL/$3 2>&1 | grep '^--' 2>&1 | awk '{ print $3 }' | grep
-v '\.\(css\|js\|png\|gif\|jpg\|JPG\)$' &Doing a bit of research on Google I've
found other people have experienced this in the past one or two years. Can
anyone shed light on the cause? Is it just wget storing the relative links it
has to traverse in main memory? Obviously there's ways to manage that but I'm
unsure if it's an issue of memory management.
Regards,Thomas Hood
- [Bug-wget] Memory Leakage,
thomas hood <=