http://www.csmonitor.com/2006/0209/p01s02-uspo.html?s=hns
(US plans massive data sweep)
Can we deny the honourable Mr. Bush from hammering our sites using
robots.txt? Based on user-agent maybe? Would they spoof? The ironic thing is
that the US government can cost Webmasters (from all over the world) some
bandwidth money...
|
