Re: ancient docs still in google (was: cygwin category of lilypond now Interpreters)

[John Mandereau]

On 2008/05/11, Valentin Villenave wrote:
> John, I'm not familiar enough with the way the docs are interfaced
> with the website, wan you help me on this? My best guess would be
> something like:
> 
> User-Agent: *
> Disallow: /@address@hidden/
> Disallow: /@address@hidden/
> Disallow: /@address@hidden/
> Disallow: /@address@hidden/
> Disallow: /@address@hidden/
> Disallow: /@address@hidden/
> 
> However, is there a way to generate this list more cleany using Python
> and some kind of a conditional loop?

Err, I don't understand why any code in any language would be needed to
do this: it's easy to write it and update it manually at each major
release, so it's not worth coding anything for this task.

If Han-Wen and Jan agree, I guess a good solution could be putting
robots.txt at root of lilypond.org, so it would contain:

Disallow: doc/v1.6/
...etc...

As an alternative, robots.txt could be put in lilypond.org/web and
contain ../doc/vX.Y/ entries, so everybody could update it in web Git
branch, but I'm not sure if search engines would find this file and
interpret '../' path correctly.

Cheers,
John