__/ [ John Bokma ] on Thursday 20 April 2006 22:36 \__
> xptomarketing@xxxxxxxxx wrote:
>
>> Hello there
>>
>> I would like to know if this following, in my awstats information is
>> normal:
>>
>> Robots/Spiders visitors (Top 25)
>>
>> Googlebot 13742 608.88 MB 20 Apr 2006 - 04:28
>>
>> MSNBot 1380 68.62 MB 20 Apr 2006 - 01:32
>> Inktomi Slurp 447 19.95 MB 20 Apr 2006 - 04:11
>> Unknown robot (identified by 'spider') 374 15.71 MB 20 Apr
>> 2006 - 01:55 Unknown robot (identified by 'crawl') 65 380.33
>> KB 19 Apr 2006 - 06:14 AskJeeves 11 215.16 KB 16 Apr
>> 2006 - 03:58 Voyager 3 283.75 KB 20 Apr 2006 - 05:06
>> WISENutbot 2 29.29 KB 05 Apr 2006 - 16:36
>> Unknown robot (identified by 'robot') 2 59.22 KB 06 Apr
>> 2006 - 18:28 Voila 2 57.23 KB 14 Apr 2006 - 17:05
>> Netcraft 1 0 06 Apr 2006 - 03:08
>>
>>
>> Google needs to "consume" 608.88 MB ?
>
> do you have images?
This aggregate includes images. But your point is a very valid one. Google
gets hungry sometimes and sometimes there is abatement. It's quite
unpredictable. A sheer example of that is Alexa (the Web Archive). This
month, MSN seems to be the 'bad guy'.
MSNBot 40296+704 1.01 GB 21 Apr 2006 - 07:53
Googlebot 34236+24 423.33 MB 21 Apr 2006 - 07:39
Inktomi Slurp 30905+2648 512.23 MB 21 Apr 2006 - 07:59
What do I get for the 1 GB of largely-on-the-fly-generated content?
MSN 58 128
At some point, I thought about kicking MSN out...
http://schestowitz.com/Weblog/archives/2005/06/27/googlebot-versus-msnbot/
> are you sure that Googlebot is *just* Googlebot?
>
> You might get a better idea when you download your access log and use:
> http://johnbokma.com/perl/googlebot-statistics.html
>
> which shows which files and how often.
>
> Using sitemap might help you.
I agree on that account!
Best wishes,
Roy
--
Roy S. Schestowitz | "Have you compiled your kernel today?"
http://Schestowitz.com | SuSE Linux ¦ PGP-Key: 0x74572E8E
9:30am up 43 days 23:13, 7 users, load average: 0.44, 0.58, 0.48
http://iuron.com - help build a non-profit search engine
|
|
