[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] Moving release tarballs to a CDN

From: Stefan Hajnoczi
Subject: Re: [Qemu-devel] Moving release tarballs to a CDN
Date: Wed, 8 Nov 2017 17:11:20 +0000
User-agent: Mutt/1.9.1 (2017-09-22)

On Wed, Nov 08, 2017 at 05:19:25PM +0100, Stefan Weil wrote:
> Am 08.11.2017 um 16:33 schrieb Stefan Hajnoczi:
> > Hi Mike and Jeff,
> > qemu.org's bandwidth usage is dominated by release tarball downloads.
> > This puts qemu.org bandwidth usage in the 2+ TB/month range.
> Hi Stefan,
> how much of this traffic is caused by web spiders?
> From my own binaries I know that the bots of the
> different search engines cause most of the traffic,
> if they are allowed to do so.
> Usually they respect robots.txt. There is no
> https://www.qemu.org/robots.txt currently.
> Nor is there a https://download.qemu.org/robots.txt.
> Adding both would reduce the downloads, maybe
> enough to fix the problem.
> Or do you see an advantage from bots which download
> QEMU tarballs? robots.txt can also block only
> selected bots.
> Regards
> Stefan
> PS. There is a https://git.qemu.org/robots.txt.

Great idea!  It's an easy to try adding a robots.txt and check how
bandwidth uses changes over the next month.

Jeff: Want to try this?


Attachment: signature.asc
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]