I personally think this whole discussion is a moot point. Maybe we
should throw a blank index.php file into wp-content to make sure that
robots can't get a directory listing, but other than that, there isn't
any information a robot would gain from spidering, other than a
WordPress login page. If users feel the need to protect themselves
more, they can add their own robots.txt.
On 11/23/05, Roy Schestowitz <r@xxxxxxxxxxxxxxx> wrote:
> _____/ On Wed 23 Nov 2005 16:46:50 GMT, [Denis de Bernardy] wrote : \_____
> >> -----Original Message-----
> >> From: wp-hackers-bounces@xxxxxxxxxxxxxxxxxxxx
> >> [mailto:wp-hackers-bounces@xxxxxxxxxxxxxxxxxxxx] On Behalf Of
> >> David Munn
> >> Sent: Wednesday, November 23, 2005 5:45 PM
> >> To: wp-hackers@xxxxxxxxxxxxxxxxxxxx
> >> Subject: [wp-hackers] Robots.txt
> >> I was thinking it would be a good idea to include a simple robots.txt
> >> file within the Wordpress release to block of bot access to the
> >> wp-admin, wp-includes, and wp-content directories.
> >> opinions?
> >> David Munn
> > make that a robots-sample.txt
> > I'd hate to loose my robots.txt by mistake.
> > D.
> Not everyone installs WordPress at the top level either. The implication of
> multiple robots.txt files would be user confusion. I suggest prepending to the
> top of each administration page e.g.
> <META HTTP-EQUIV="CACHE-CONTROL" CONTENT="NO-CACHE">
> <META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">
> Roy S. Schestowitz
> http://Schestowitz.com | SuSE Linux | PGP-Key: 0x74572E8E
> 4:15am up 21 days 0:09, 4 users, load average: 0.18, 0.33, 0.26
> wp-hackers mailing list
wp-hackers mailing list