[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#52338: Crawler bots are downloading substitutes
|
From: |
Leo Famulari |
|
Subject: |
bug#52338: Crawler bots are downloading substitutes |
|
Date: |
Fri, 10 Dec 2021 11:22:15 -0500 |
On Thu, Dec 09, 2021 at 04:42:24PM +0100, Tobias Geerinckx-Rice wrote:
[...]
> An alternative to that is to serve a real on-disc robots.txt.
Alright, I leave it up to you. I just want to prevent bots from
downloading substitutes. I don't really have opinions about any of the
details.
signature.asc
Description: PGP signature
bug#52338: Crawler bots are downloading substitutes, Mark H Weaver, 2021/12/10