Bots should not be able to get to properly password-protected content.<div><br></div><div>You can tell bots to avoid specific paths by adding "Disallow: /path/" lines to your robots.txt.</div><div><br></div><div>
Brian</div><div><br></div><div><br></div><div><br><div class="gmail_quote">On Fri, Dec 4, 2009 at 2:45 AM, Armin Medosch <span dir="ltr"><<a href="mailto:armin@easynet.co.uk">armin@easynet.co.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Hi<br>
<br>
I am running drupal 5.x and og with panels. We recently started a<br>
research group which is protected by password from public, but I think<br>
google and other search index bots are still able to index those pages.<br>
<br>
What do I need to put into robots.txt to stop the bots indexing this<br>
specific og group?<br>
<br>
thanks and best regards<br>
Armin<br>
<br>
<br>
--<br>
thenextlayer software, art, politics <a href="http://www.thenextlayer.org" target="_blank">http://www.thenextlayer.org</a><br>
<font color="#888888"><br>
<br>
--<br>
[ Drupal support list | <a href="http://lists.drupal.org/" target="_blank">http://lists.drupal.org/</a> ]<br>
</font></blockquote></div><br>
</div>