<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
This may seem like a lot of work for solving your issue, but depending
on how desperate you are it should work. You could right a python
script that digs through the log files and skips all the bogus lines
that match a certain pattern and writes the lines that don't fit that
pattern to a new log file. The end result would be a new log file that
contains only the information you need.<br>
<br>
Mike<br>
<br>
<div class="moz-signature"><a href="http://www.getfirefox.com/"><img
alt="Firefox 2" title="Firefox 2"
src="cid:part1.07040003.09060909@gmail.com" border="0"></a></div>
<br>
<br>
Tibor Liktor wrote:
<blockquote cite="mid:20070830151157.3c2bbff0@szerver.itthon"
type="cite">
<pre wrap="">On Thu, 30 Aug 2007 08:11:19 -0400
Earnie Boyd <a class="moz-txt-link-rfc2396E" href="mailto:earnie@users.sourceforge.net"><earnie@users.sourceforge.net></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Quoting Tibor Liktor <a class="moz-txt-link-rfc2396E" href="mailto:liktor@gmail.com"><liktor@gmail.com></a>:
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
I've got a watchdog problem.
The watchdog log is essential for me to discover bugs and errors, and
monitor the site's
performance, and blahblablah - you know that.
But watchdog became quite useless for me, because it is full of 404
errors triggered by Google and
other bots.
Now nearly 90% of the log is crap. It is impossible to dig out any
useful info from that. (Not
speaking about the additional server load and database size issues.)
Is there any solution to filter out the tons of messages caused by
searchbots?
Do you face with similar issues? How do you handle those?
</pre>
</blockquote>
<pre wrap="">There is a filter by message type list box that can help but that is
too simple and I must not be understanding you. I do know that the
"page not found" errors tend to be ridiculous and the drupal engine
itself causes a several; i.e. the referrer is the site on which the
watchdog log is being reviewed. It would be nice if I could filter to
"all messages except page not found messages"; is that what you mean?
Can't do it out of the box but that doesn't mean you can't program for
it.
Earnie -- <a class="moz-txt-link-freetext" href="http://for-my-kids.com/">http://for-my-kids.com/</a>
-- <a class="moz-txt-link-freetext" href="http://give-me-an-offer.com/">http://give-me-an-offer.com/</a>
</pre>
</blockquote>
<pre wrap=""><!---->
Hi,
no, I need the "normal" 404 messages, since they provide me useful infos about broken links on the
site.
My problem is the Googlebot tries links thousand years dead, etc. and generates a massive amount of
unnecessary entries in my watchdog.
I only would like to filter for domainnames, IP-s, etc.
Best,
Tibor
</pre>
</blockquote>
</body>
</html>