[Axiom-developer] RE: search engines accessing MathAction websites

[Bill Page]

I wrote:

> 
> So today, I made url specific urls that block all robots from the
> first two urls but allow full indexing of both the wiki and the
> portal (except for bibliography and SandBox pages):
> 
> User-agent: *
> Disallow: /SandBox
> Disallow: /bibliography
> Disallow: /refs/axiombib
> Disallow: /refs/redbib
> 

I meant:

  So today, I made url specific robots.txt files that block all
  robots from the ...