[support] Getting Control of Spiders...

Support folk, I'm getting Page not Found errors in my Drupal log every hour, on multiple sites on my server, for the files I deleted from the root directory: - install.php - CHANGELOG.txt - INSTALL.txt - INSTALL.mysql.txt - INSTALL.pgsql.txt - LICENSE.txt - MAINTAINERS.txt - UPGRADE.txt - COPYRIGHT.txt The robots.txt tells the bots to ignore those pages... but??? Should I just ignore it? Any good wisdom someone has on advising on how to handle all those bots that are pinging my server all the time? Thanks, Shai