Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: Googlebot Gone Crazy

___/ On Friday 26 August 2005 06:49, [Alan Cole] wrote : \___

> Is there any way to stop the Googlebot visiting a site quite so
> often.... I know a robots.txt file can restrict their access to certain
> pages but I don't want to do that as I still want all pages listed, I'd
> just rather the Googlebot didn't visit so often as it eats up all of my
> bandwidth.
> 
> It seems to visit my site every few days and manages to crawl through
> something like 30,000 pages each time it does so.
> 
> Thanks.
> Al.


http://www.google.com/bot.html#fast

Hope it helps,

Roy

-- 
Roy S. Schestowitz        chmod a-r *.mbox
http://Schestowitz.com

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index