Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: Maybe BigDaddy is Here to Stay...

__/ [ Paul ] on Monday 12 June 2006 09:17 \__

> On Mon, 12 Jun 2006 04:12:56 +0100, Roy Schestowitz
> <newsgroups@xxxxxxxxxxxxxxx> wrote:
> 
>>http://www.webworkshop.net/google-madness.html
>>
>>,----[ Snippet ]
>>| In the first few months of this year (2006), Google rolled out a new
>>| infrastructure for their servers. The infrastructure update was called
>>| "Big Daddy". As the update was completed, people started to notice that
>>| Google was dropping their sites' pages from the index - their pages were
>>| being dumped. Many sites that had been fully indexed for a long time were
>>| having their pages removed from Google's index, which caused traffic to
>>| deteriorate, and business to be lost. It caused a great deal of
>>| frustration, because Google kept quiet about what was happening.
>>| Speculation about what was causing it was rife, but nobody outside
>>| Google knew exactly why the pages were being dropped.
>>| 
>>| Then on the 16th May 2006, Matt Cutts, a senior Google software
>>| engineer, finally explained something about what was going on. He
>>| said that the dropping of pages is caused by the improved crawling
>>| and indexing functions in the new Big Daddy infrastructure, and he
>>| gave some examples of sites that had had their pages dropped.
>>`----
>>
>>I hope that others will find this informative.
> 
> If this is all true, then soon there will be no need for google site
> maps or google search (as in adsense)

I don't know if it means anything, but Google's crawling is on the rise in my
main domain.

Last month (31 days):

Inktomi Slurp   47119+6604      798.23 MB       31 May 2006 - 23:24
Googlebot       30191+40        364.55 MB       31 May 2006 - 23:54
MSNBot  28031+956       446.40 MB       31 May 2006 - 23:55

This month (~11 days):

Googlebot       25961+17        253.87 MB       12 Jun 2006 - 06:02
Inktomi Slurp   19708+1596      323.06 MB       12 Jun 2006 - 06:11
MSNBot  7901+304        108.86 MB

Can you spot a similar trend in any of your sites? As for cached/indexed
pages, there is a _miserably_ slow increase (the scale of 0.1% per day)...

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index