Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: how to stop crawler to index web page?

__/ [ z ] on Saturday 23 September 2006 17:52 \__

> Roy Schestowitz wrote:
> 
>> __/ [ Paul ] on Saturday 23 September 2006 12:13 \__
>> 
>>> On Sat, 23 Sep 2006 12:08:37 +0100, Roy Schestowitz
>>> <newsgroups@xxxxxxxxxxxxxxx> wrote:
>>> 
>>>>Could sniff user-agent and deliver different content
>>> 
>>> Doesn't that come under black hat ?
>> 
>> Maybe for the Web site (well, it's the Webmaster that risks banishment),
> 
> Unless you are buddies with Google, like Webmasterworld.  I still can't
> believe that Google lets them cloak their pages like that.  Try visiting as
> Googlebot and watch the login screen disappear and the link structure
> change.

I guess that the NYT (among others) should not get any Google luv either.
Compromises, compromises.

For some reason, I seem to recall a rumour/article which said googleguy is
Matt Cutts. Among other insiders in other forums...

Best wishes,

Roy

-- 
Roy S. Schestowitz      | Linux + tax = Mac OS = (Windows - functionality)
http://Schestowitz.com  | Free as in Free Beer ¦  PGP-Key: 0x74572E8E
Cpu(s):  19.5% user,   2.7% system,   0.9% nice,  76.9% idle
      http://iuron.com - semantic engine to gather information

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index