__/ [Ignoramus25589] on Friday 14 October 2005 05:31 \__
> It crawls my websites algebra.com and cooldictionary.com way too
> fast.
>
> Today, for instance, algebra.com received 67,000 accesses from
> googlebot.
>
> They also misbehave and do not honor Crawl-delay setting.
>
> I am not going to ban googlebot from crawling my site, but I am
> becoming somewhat apprehensive of them.
>
> I want to limit their accesses to, say, no more than every 5
> seconds. How can I do it, realistically speaking.
>
> i
{
* Nice to have you back in this NG
* Mikkel, I am aware of the subject line which will be bound to my name in
SEPR's... ouch!
}
Someone in this newsgroup asked a similar question a month or two back. It
is not quite possible to limit the demands that reach from Google if I
recall correctly. That was the conclusion reached, I think. My recollection
tells me that the best move would be to contact Google saying that your
dedicated server is knocked on too hard.
(since I bound my name to the subject line, I might as well go all the way)
001 650 330 0100
Call them to get an E-mail address and then be in touch with a real person
promptly.
Hope it helps,
Roy
--
Roy S. Schestowitz | Have you hugged your penguin today?
http://Schestowitz.com | SuSE Linux | PGP-Key: 74572E8E
6:10am up 49 days 18:24, 4 users, load average: 0.82, 0.84, 0.73
http://iuron.com - next generation of search paradigms
|
|
