[Mldonkey-bugs] [Bug #1555] Ressource starvation with high source files causes several serious bugs

[nobody]

=================== BUG #1555: FULL BUG SNAPSHOT ===================
http://savannah.nongnu.org/bugs/?func=detailbug&bug_id=1555&group_id=1409

Submitted by: drax                      Project: mldonkey, a free e-Donkey 
client
Submitted on: 2002-Oct-30 12:07
Category:  eDonkey-plugin               Severity:  5 - Major                    
Bug Group:  None                        Resolution:  None                       
Assigned to:  None                      Status:  Open                           
Release:  2.00                          Release:                                
Platform Version:  Linux i386-i686      Binaries Origin:  Downloaded from 
Savannah

Summary:  Ressource starvation with high source files causes several serious 
bugs

Original Submission:  Hello all, I think I found the bug which causes the 
following problems:

- very high CPU load
- very high memory load
- upstream limit exceeded
- high performance drop

In fact this all stems from the very same single bug. If you add a "high 
source" file to mldonkey (ie a file with 1000+ sources) then mldonkey starts to 
continously find all these sources and get their chunk list. This traffic WILL 
surpass any upstream limit and cause performance degradation if it saturates 
your upstream.

Also the many many sources mldonkey is continously asking will also cause a 
very high CPU and memory load. This behavour is INDEPENDENT of any value of 
max_sources_per_file (I've set max_sources_per_file to 500 and still mldonkey 
contacts far too many peers. It seems it tries to open up max_sources_per_file 
ACTIVE, OPEN connections. Also it always searches for more peers for that file)

Setting max_opened_connections seems not to help either (I've set it to 600 and 
still ressource starvation on high source files)

I've attached an image to this report. There you will see that MLDonkey limits 
file transfer upstream perfectly to 5 KBytes per second, but total upstream is 
higher because of the constant peer-contacting. In my case, this 
meta-upstreamtraffic is 9 Kbytes/sec ! Also the return traffic of this 
non-filetransfer traffic is around the same magnitude. So this bug does not 
only waste unneccessary CPU cycles and memory but also quite a huge amount of 
bandwith.

Please note that I don't have a T-DSL connection, my connection is capped at 
hard 512'000/128'000 bits per second and it is not xDSL, so no pppoe overhead. 
Max upstream I've ever reached is 15.4 KBytes/sec but at about 13.5 KBytes 
downstream starts to collapse.

Also you can very easily see in the attached picture, that the MLDonkey GUI 
reports the current file transfer up/down speed which is limited very nicely to 
what it should and you can also see how much really is transfered in total, 
including the constant queries and answers of houndreds of peers.

Also you can see the quite high memory usage. It grows over time (but sometimes 
falls back too). Letting MLDonkey running over night in this configuration 
cause my Linux to kill the task in the early morning due to it eating too much 
memory:
Oct 30 06:11:28 linux kernel: VM: killing process screen
(mldonkey was started using screen)

Workaround: Pausing the high-source files does NOT help. It might but after 1 
hr after the high-source file got paused I still had upstream exceeding 
connection traffic. Maybe the backlog gets too high and would settle down after 
a long time.

Working Workaround: Pause the high-source file(s). Kill mldonkey. Get the 
file-hash of the high-source files and remove them from your mldonkey temp dir. 
Extract the entries for those files from files.ini. Restart mldonkey without 
those files. Retry later when less sources exist or when bug is fixed.





No Followups Have Been Posted


CC list is empty


File Attachments
****************

-------------------------------------------------------
Date: 2002-Oct-30 12:07  Name: mld.jpg  Size: 250KB   By: drax
MLdonkey Screenshots ressource starvation
http://savannah.nongnu.org/bugs/download.php?group_id=1409&bug_id=1555&bug_file_id=127


For detailed info, follow this link:
http://savannah.nongnu.org/bugs/?func=detailbug&bug_id=1555&group_id=1409