star green wrote:
> Ive been trying to figure out how to write a robots.txt file that will
> allow the robots to access the homepage (index.html), but no other page on
> the site.
>
> Short of putting in a "disallow" line for every webpage on the site,
> except for the homepage, is there some easy way to do this? I've been to
> several webpages about robots.txt but haven't been able to come up with
> anything.
>
> any advice is appreciated!
For the record, if your site is hierarchically built, it should also be
possible to disallow access to the page higher up in the 'roots' of the
site. The only problem might then be direct links that point deep into your
Web site.
Roy
--
Roy Schestowitz
http://schestowitz.com
|
|
