[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: robots.txt in git and online are not the same

From: Mark Polesky
Subject: Re: robots.txt in git and online are not the same
Date: Sun, 30 Jun 2013 05:58:32 -0700 (PDT)

If robots.txt was getting updated properly, all of our Google
search bar problems would be solved.  We could then stop telling
Google to restrict the search results to a patrticular version
from the search box itself.  The robots.txt file only allows the 
current stable docs to be indexed.  

If robots.txt was getting updated properly, then only 2.16 should
appear in search results, and would continue to appear for a
little while after we release 2.18.  Then, the next time the 
Google bot visits lilypond.org, it would read robots.txt, remove
the 2.16 docs from the search results, then follow every link from
the homepage that isn't "disallowed", i.e. the entire website,
with only the 2.18 docs.  That's exactly what we want, so let's
just fix the problem with robots.txt.

By the way, fixing that would kill 3 items in the tracker with one 

Issue 2909: Manual search returns results from wrong version

Issue 3209: Searching stable release documentation should only return results 
from stable release

Issue 3367:   Web/Docs: LilyPond version is not clear on docs web pages

- Mark

----- Original Message -----
> From: Mark Polesky <address@hidden>
> To: bug-lilypond <address@hidden>
> Cc: 
> Sent: Friday, June 28, 2013 11:37 AM
> Subject: robots.txt in git and online are not the same
> For some reason, http://www.lilypond.org/robots.txt differs
> from Documentation/web/server/robots.txt as found in git.  
> It looks as if it isn't being updated on the server?  I 
> don't get it, but it's worth looking into, since this helps
> remove outdated links when someone googles us from the
> outside.
> - Mark
> _______________________________________________
> bug-lilypond mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/bug-lilypond

reply via email to

[Prev in Thread] Current Thread [Next in Thread]