[afnog] Website robots.txt prevents indexing

Chris Wilson chris+afnog at aptivate.org
Tue Oct 1 14:27:18 UTC 2013


Hi all,

Why do we not want Google indexing the www.ws.afnog.org website?

http://www.ws.afnog.org/robots.txt says:

 	# go away
 	User-agent: *
 	Disallow: /

There's no other way to search the website, and some kind of keyword search 
would be really useful.

Cheers, Chris.
-- 
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.




More information about the afnog mailing list