__/ [ ato_zee@xxxxxxxxxxx ] on Sunday 22 October 2006 20:47 \__
>
> On 22-Oct-2006, "Lars Bonnesen" <none@none.æøå> wrote:
>
>> Is there a commen way to temp. disable indexing and followings on these
>> search engine crawlers?
>
> Google for robots.txt
>
> Many references. You can have different folders on your server
> for testing a new site build, and use robots.txt to exclude the
> search engines. The robots.txt has to go in the root directory.
Here's a golden reference... one that I used years ago.
http://www.robotstxt.org/wc/robots.html
Hope it helps,
Roy
--
Roy S. Schestowitz, Ph.D. Candidate (Medical Biophysics)
http://Schestowitz.com | SuSE Linux | PGP-Key: 0x74572E8E
9:05pm up 4 days 6:19, 8 users, load average: 0.33, 0.46, 0.52
http://iuron.com - Open Source knowledge engine project
|
|
