[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] Moving release tarballs to a CDN

From: Stefan Weil
Subject: Re: [Qemu-devel] Moving release tarballs to a CDN
Date: Wed, 8 Nov 2017 17:19:25 +0100
User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0

Am 08.11.2017 um 16:33 schrieb Stefan Hajnoczi:
> Hi Mike and Jeff,
> qemu.org's bandwidth usage is dominated by release tarball downloads.
> This puts qemu.org bandwidth usage in the 2+ TB/month range.

Hi Stefan,

how much of this traffic is caused by web spiders?

>From my own binaries I know that the bots of the
different search engines cause most of the traffic,
if they are allowed to do so.

Usually they respect robots.txt. There is no
https://www.qemu.org/robots.txt currently.
Nor is there a https://download.qemu.org/robots.txt.
Adding both would reduce the downloads, maybe
enough to fix the problem.

Or do you see an advantage from bots which download
QEMU tarballs? robots.txt can also block only
selected bots.


PS. There is a https://git.qemu.org/robots.txt.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]