> On Wed, 12 Jan 2005 22:56:54 GMT, "star green" <firstname.lastname@example.org>
>>Ive been trying to figure out how to write a robots.txt file that will
>>allow the robots to access the homepage (index.html), but no other page on
>>Short of putting in a "disallow" line for every webpage on the site,
>>except for the homepage, is there some easy way to do this?
> One suggestion: Could put the other pages into a subdirectory folder
> then disallow the spiders from those folders.
This can be a big overhaul in the absence of _relative_ links.
I used the following excellent source for all my robots.txt needs: