Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: robots.txt

__/ [ Borek ] on Friday 21 April 2006 08:20 \__

> On Fri, 21 Apr 2006 04:15:34 +0200, Roy Schestowitz
> <newsgroups@xxxxxxxxxxxxxxx> wrote:
> 
>> I don't want to step on anybody's toes, but the one-stop place for quick
>> reference is probably:
>>
>>         http://www.robotstxt.org/wc/robots.html
>>
>> It explains everything that is supported and does so fairly well.
>> Fortunately, no search engines have yet diverged from the standard by
>> supporting wildcards.
> 
> I thought Google did.

There was a blurb about it last month, but I thought it was all just noise.
Is it true??? Is it a 'one-way street innovation' (one to benefit self with
the exclusion of others), much like XML sitemaps?

Best wishes,

Roy

-- 
Roy S. Schestowitz      | Linux + tax = Mac OS = (Windows - functionality)
http://Schestowitz.com  |    SuSE Linux    ¦     PGP-Key: 0x74572E8E
  8:25am  up 43 days 22:08,  7 users,  load average: 0.56, 0.80, 0.78
      http://iuron.com - next generation of search paradigms

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index