[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] Benchmarking
From: |
Tim Rühsen |
Subject: |
Re: [Bug-wget] Benchmarking |
Date: |
Sat, 14 Sep 2019 23:30:47 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 |
Hi Marcel,
wget1.x can only download one file at a time. Of course you can use
multiple instances in parallel.
Wget2 uses 5 parallel threads per default (can be tuned with
--max-threads=N), so no need for parallel instances. If you give e.g. 5
or less URLs, they are downloaded in parallel without the need for tuning.
Here comes a little complication... HTTP/2 behaves slightly different
than HTTP/1.1 as it allows parallel requests (streams) on one connection
- if all URLs have the same domain.
Next, Wget2 has a stats module that allows to generate one data record
per URL including timings (millisecond precision), see --stats-site.
In the Wget2 repository are also some benchmark scripts in subdir
benchmarks/.
Example:
$ wget2 --stats-site=csv:- www.gnu.org www.heise.de www.oracle.de
...
ID,ParentID,URL,Status,Link,Method,Size,SizeDecompressed,TransferTime,ResponseTime,Encoding,Verification,Last-Modified,Content-Type
1,0,http://www.gnu.org,200,1,1,9168,28592,235,234,1,0,0,text/html
2,0,https://www.heise.de,200,1,1,58158,434935,53,27,1,0,1568496351,text/html
3,0,http://www.oracle.de,301,1,1,217,270,341,341,1,0,0,text/html
4,0,http://www.oracle.com/de/index.html,301,0,1,0,0,180,180,0,0,0,(null)
5,0,https://www.oracle.com/de/index.html,200,0,1,8511,32215,160,160,1,0,1568450074,text/html
It also records the redirections as you can see. See docs/wget2.md for
more info regarding the fields.
Regards, Tim
On 14.09.19 17:33, Schaible, Marcel wrote:
> Hi,
>
>
> I am wondering if wget is capable of being used as a benchmark tool for web
> servers.
>
>
> I just come back from a programming course for beginners and we have setup
> for the attendees a small webserver (nginx) as download hub for some
> installation packages. The server was dead slow and practically unusable. For
> the next course I want to test this upfront.
>
>
> So I am looking for some wget options to start a couple of concurrecnt
> connections to my server and download files, preferable a set of different
> files.
>
>
> Can this be done with wget.
>
>
> Thanks
>
>
> Marcel
>
signature.asc
Description: OpenPGP digital signature