On Wed, 12 Jan 2005 22:56:54 GMT, "star green" <spam@verizon.net>
wrote:
>Ive been trying to figure out how to write a robots.txt file that will allow
>the robots to access the homepage (index.html), but no other page on the
>site.
>
>Short of putting in a "disallow" line for every webpage on the site, except
>for the homepage, is there some easy way to do this?
One suggestion: Could put the other pages into a subdirectory folder
then disallow the spiders from those folders.
Carol
|
|
