__/ [Carol W] on Tuesday 08 November 2005 01:39 \__
> On Thu, 03 Nov 2005 16:55:29 GMT, info_at_1-script_dot_com@xxxxxxx
> (www.1-script.com) wrote:
>
>>Roy Schestowitz wrote:
>>
>>
>>
>>> I have noticed that MSN and Yahoo index less than 5% of the pages in
>>> my
>>> site while Google is 'aware' of the existence of them all. I tried
>>> to
>>> think if MSN and Yahoo are hindered by JavaScript, but there
>>> are
>>> JavaScript-free routes to all pages.
>>
>>> Why is it that Yahoo and MSN will not crawl deep or at least index
>>> pages
>>> that are buried deeper in the site? Judging by the logs, they
>>> consume
>>> (spider) just about as much traffic as Google. When it comes to
>>> referrals,
>>> Google delivers almost 100 times the amount of referrals from Yahoo
>>> and
>>> MSN combined. How can this be?
>>
>>> Help appreciated,
>>
>>> Roy
>>
>>Looking at last 7 days data, I got 95% of SE traffic from Google, 2.2%
>>from Yahoo and 0.6% from MSN.
>>Yet, the downloaded amount data goes like this:
>>Yahoo! Slurp: 603.6Mb
>>msnbot: 278.5MB
>>Googlebot (Freshbot):112.7MB
>>Googlebot (Deepbot): 155.9Mb
>>
>>The only way I can see this happening is that Y and MSN do not deem my
>>pages important enough to include in the indices, but still important
>>enough to keep banging on them.
>
> I can't talk about MSN but, in my past observations, Yahoo is pretty
> slow in updating their index.
That seems plausible. I wonder how they survive if that's true... maybe an
improper search engine for anything recent.
> One time I posted here that Yahoo kept showing a cache copuy of one of
> my pages that was over a year old - even though the site design had
> been changed (drastically) twice within that year - and some of the
> contents were changes as well. In terms of new pages, I don't hold my
> breath with Yahoo. Seen it take as long as 6 to 9 months before my
> "new" pages are finally part of the index there. Wherease Google
> reflects changes or new pages within a few short hours.
>
> No idea why Yahoo is so slow on refreshing their index, or even the
> cahce, as slurpbot is rather active in visiting sites so takes note
> of new pages and changed contents but that apparently is how it is
> with Yahoo .. and I can't see how MSN could do any worse.
>
> Carol
I just keep wondering why both MSN and Yahoo stop at roughly the same
threshold of 2000 pages. If they were both out-of-date, would the number of
pages be so closely related? I am sure nothing is negotiated among the two
and I would like to believe that what you say is true. If Yahoo and MSN will
index more pages (*and* display them) in due time, then I truly look forward
to it.
Roy
--
Roy S. Schestowitz
http://Schestowitz.com | SuSE Linux | PGP-Key: 0x74572E8E
5:25am up 5 days 1:23, 5 users, load average: 0.13, 0.22, 0.31
http://iuron.com - next generation of search paradigms
|
|
