Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: how to stop crawler to index web page?

Roy Schestowitz wrote:

> __/ [ z ] on Saturday 23 September 2006 17:52 \__
> 
>> Roy Schestowitz wrote:
>> 
>>> __/ [ Paul ] on Saturday 23 September 2006 12:13 \__
>>> 
>>>> On Sat, 23 Sep 2006 12:08:37 +0100, Roy Schestowitz
>>>> <newsgroups@xxxxxxxxxxxxxxx> wrote:
>>>> 
>>>>>Could sniff user-agent and deliver different content
>>>> 
>>>> Doesn't that come under black hat ?
>>> 
>>> Maybe for the Web site (well, it's the Webmaster that risks banishment),
>> 
>> Unless you are buddies with Google, like Webmasterworld.  I still can't
>> believe that Google lets them cloak their pages like that.  Try visiting
>> as Googlebot and watch the login screen disappear and the link structure
>> change.
> 
> I guess that the NYT (among others) should not get any Google luv either.
> Compromises, compromises.

Yeah... NYT is another one.  I haven't tested their site as Googlebot.

What if every web site started cloaking like this?  Visitors would have to
login every time they click on a result in the SERPs.  It's not right.  If
webmasters want their site in the search engines they shouldn't be allowed
to hide it behind a "members only" screen.

There was another webmaster forum that did something like this.  They were
at the top of Google for many tech queries but when you got to the site you
had to register to find the answer.  I think there was even a Firefox
extension that removed those results from the SERPs.  Was it Sitepoint in a
earlier incarnation?  All I can remember is the awful "Register to view the
solution" screen...


> For some reason, I seem to recall a rumour/article which said googleguy is
> Matt Cutts. Among other insiders in other forums...

No surprise.  Looks like a textbook case of croynism.


[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index