Roy Schestowitz <newsgroups@xxxxxxxxxxxxxxx> wrote:
>* Volume of crawling, expressed in terms of bandwidth (most easily digestable
>as it's a single number that contains many pertinent details)
I assume you're talking number of bytes sent to each robot over some
time period?
>* Breakdown into crawler names (vendors). This enables the Webmaster to
>assess how SEO affects the different algorithms. It specifically tells the
>Webmaster what appeals for more attention, crawling and activity (often
>backlinks).
Got that already I think.
>Last page fetched is irrelevant if general
>traffic is high and the number of pages is high.
It can be relevant when you are adding new pages, I think. For (a
slightly different) example, if I blow away the data for google then
look a bit later, the times on pages can tell which were fetched in
what order, which tells a lot about algorithm. However currently I'm
only keeping last-fetched time so multiple fetches invalidate
conclusions drawn here.
>I also cannot see how IP
>addresses of robots contribute anything.
If you decide to block a particular robot on ip-address it can be
helpful to know what ip-addresses, I think.
>> [And if anyone can tell me why I keep typing "robit" instead of
>> "robot" that would be greatly appreciated!]
>
>
>Typing pattern habits as in "weird/wierd" or maybe a Freudian slip (Rabbit?).
I think it's mostly a case of pre-coffee fatfingering. Thanks Roy.
--
http://www.ren-prod-inc.com/hug_soft/store.php?action=contact
|
|
