It appears that the MSN spider practically lives at my web site.

My watchdog logs are filling up with requests to non-existent pages, pages that are member-only (access denied errors) and the like.

The culprit is at a narrow range of Microsoft IP's.

207.46.98.129 - 207.46.98.134

Unrelated, I have some kind of bot that keeps trying to hit a (non-existent) page at: node/14

This is a varying address, but always in the same IP block (not the same IP). The attempt is made (to this and other places) about 6 to 8 times per day, usually with 10 or so attempts at various "admin/*" pages along the way.

What tools are others using to limit this bandwidth-sucking and log-clogging activity?

Cheers, -- Gary

P.S. ...for those with a sense of humor...

Bad Behavior seemed reasonable, but the saga of my trying it out today goes, so far, something like this:

I installed this on my localhost server and on a live server, and I get the same error in both environments.

This is the same error reported here:

http://www.angier.co.uk/drupal/downloads

I got my copy from the 4.6.x link here:

http://www.angier.co.uk/drupal/downloads

I followed the installation instructions here:

http://www.angier.co.uk/drupal/instructions/drupal-badbehavior-122

Which lead me to get another copy from here:

http://www.ioerror.us/software/bad-behavior/bad-behavior-download/

Which produces the same error. Okay, it's me.

I now have a 'bad-behavior' v1.2.4 download, whose ReadMe.txt file says:

This README.txt file applies to WordPress installations. For other types of installations, please see the documentation at

http://www.ioerror.us/software/bad-behavior/installing-and-using-bad-behavi or/

Which then tells me to go here:

http://www.ioerror.us/software/bad-behavior/installing-and-using-bad-behavi or/on-drupal/

Which says:

"The installation process for Bad Behavior for Drupal is a little different from other software on which Bad Behavior runs. You will need to download Bad Behavior for Drupal and be sure to follow the installation directions given there."

With a download (but not really) link to:

http://www.angier.co.uk/david/2005-08-19-drupal-bad-behaviour

Which I follow, and which then tells me:

"So, this is a fairly complete implementation, ready to roll. Download here."

With a (not yet, really) download link to:

http://www.angier.co.uk/drupal/downloads

And which also goes on to tell me:

"To install, unpack Bad Behavior from here into a sub-directory of the modules directory and then copy badbehaviour.module into that folder."

With 'here' being linked to:

http://www.ioerror.us/2005/08/16/bad-behavior-12/

So, split over two sites, 3 blogs and 4 interfaces, the author of this software still hasn't informed me how to install and evaluate the software.

Whew.

I'm still trying to crack the code that is the 'bad behavior' module.

And it will probably turn out to be simple, or "me", but how can a software allow such spiraling uncertainty from a user? A simple paragraph from the author, and a single source of information, would probably tell me what I need to do. That part, at least, can not be considered "user error".

I shall prevail!

Roses are #ff0000 Violets are #0000ff All my base are belong to you.

-- Mwa ha ha!