[Savannah-cvs] [MailSystem] (edit) explain why ML archives are not indexed.

[Beuc]

++added:
Note: archives are currently not indexed by search engines, see robots.txt::

 User-agent: *
 Disallow: /

This is an effort from address@hidden to reduce the disk I/O on this already 
loaded machine. Spiders tend to put lists.gnu.org to its knees :/

It is suggested to use mail archives mirroring services meanwhile. Once there 
is a more powerful machine, maybe web crawlers will be allowed again.


--
forwarded from 
https://savannah.gnu.org/maintenance/address@hidden://savannah.gnu.org/maintenance