Watchdog dieing on large queries
I've found a workaround for myself for this one but just wanted to put it out there. When a query fails watchdog is used to insert it into the DB, which is fine 99.9% of the time, until that .1% bites you. The problem is that if the query (say inserting a new record into the "cache") causes a query that is larger than the MySQL query buffer limit. This results in a MySQL error that the query is too large to insert, which watchdog then picks up and tries to insert, which fails, which watchdog picks up ... and you get the point. Result? Infinite loop and then either a timeout or PHP runs out of memory and dies. I'd think the proper way to handle this would be to have a check that the specific error code for a MySQL error resulting from a query buffer violation doesn't try to insert the query, again just putting this out there.
sounds reasonable. please file an issue and ideally provide a patch. On Wed, Apr 22, 2009 at 12:12 PM, Steven Surowiec <steven@digitalpulp.com> wrote:
I've found a workaround for myself for this one but just wanted to put it out there. When a query fails watchdog is used to insert it into the DB, which is fine 99.9% of the time, until that .1% bites you. The problem is that if the query (say inserting a new record into the "cache") causes a query that is larger than the MySQL query buffer limit. This results in a MySQL error that the query is too large to insert, which watchdog then picks up and tries to insert, which fails, which watchdog picks up ... and you get the point. Result? Infinite loop and then either a timeout or PHP runs out of memory and dies. I'd think the proper way to handle this would be to have a check that the specific error code for a MySQL error resulting from a query buffer violation doesn't try to insert the query, again just putting this out there.
On Wed, Apr 22, 2009 at 12:12 PM, Steven Surowiec <steven@digitalpulp.com>wrote:
I've found a workaround for myself for this one but just wanted to put it out there. When a query fails watchdog is used to insert it into the DB, which is fine 99.9% of the time, until that .1% bites you. The problem is that if the query (say inserting a new record into the "cache") causes a query that is larger than the MySQL query buffer limit. This results in a MySQL error that the query is too large to insert, which watchdog then picks up and tries to insert, which fails, which watchdog picks up ... and you get the point.
This is why we have syslog as an option in core since D6. Having to log to the database does not scale for may sites, either because they are large and hit the db too often, or because of large queries that overflow MySQL buffer or something else. I used it on most D6 sites.
Result? Infinite loop and then either a timeout or PHP runs out of memory and dies. I'd think the proper way to handle this would be to have a check that the specific error code for a MySQL error resulting from a query buffer violation doesn't try to insert the query, again just putting this out there.
Seems like a reasonable change to the dblog module. -- Khalid M. Baheyeldin 2bits.com, Inc. http://2bits.com Drupal optimization, development, customization and consulting. Simplicity is prerequisite for reliability. -- Edsger W.Dijkstra Simplicity is the ultimate sophistication. -- Leonardo da Vinci
There's another kind of error I'm dealing with today, It's not a too long inserting error, but php extension errors would do.. fsockopen for example starts another dblog loop when the connection fails because of drupal's inhability to parse the error (the unserialize on dblog.admin.inc, line:251 -> return t($dblog->message, unserialize($dblog->variables)); I just noticed it, but did not debug enough about. On Wed, Apr 22, 2009 at 7:00 PM, Khalid Baheyeldin <kb@2bits.com> wrote:
On Wed, Apr 22, 2009 at 12:12 PM, Steven Surowiec <steven@digitalpulp.com>wrote:
I've found a workaround for myself for this one but just wanted to put it out there. When a query fails watchdog is used to insert it into the DB, which is fine 99.9% of the time, until that .1% bites you. The problem is that if the query (say inserting a new record into the "cache") causes a query that is larger than the MySQL query buffer limit. This results in a MySQL error that the query is too large to insert, which watchdog then picks up and tries to insert, which fails, which watchdog picks up ... and you get the point.
This is why we have syslog as an option in core since D6. Having to log to the database does not scale for may sites, either because they are large and hit the db too often, or because of large queries that overflow MySQL buffer or something else.
I used it on most D6 sites.
Result? Infinite loop and then either a timeout or PHP runs out of memory and dies. I'd think the proper way to handle this would be to have a check that the specific error code for a MySQL error resulting from a query buffer violation doesn't try to insert the query, again just putting this out there.
Seems like a reasonable change to the dblog module. -- Khalid M. Baheyeldin 2bits.com, Inc. http://2bits.com Drupal optimization, development, customization and consulting. Simplicity is prerequisite for reliability. -- Edsger W.Dijkstra Simplicity is the ultimate sophistication. -- Leonardo da Vinci
I also have this problem. Any Solution?? It caused me a timeout and then "MySQL server has gone away" response message... -----Original Message----- From: development-bounces@drupal.org [mailto:development-bounces@drupal.org]On Behalf Of Steven Surowiec Sent: Wednesday, April 22, 2009 7:12 PM To: development@drupal.org Subject: [development] Watchdog dieing on large queries I've found a workaround for myself for this one but just wanted to put it out there. When a query fails watchdog is used to insert it into the DB, which is fine 99.9% of the time, until that .1% bites you. The problem is that if the query (say inserting a new record into the "cache") causes a query that is larger than the MySQL query buffer limit. This results in a MySQL error that the query is too large to insert, which watchdog then picks up and tries to insert, which fails, which watchdog picks up ... and you get the point. Result? Infinite loop and then either a timeout or PHP runs out of memory and dies. I'd think the proper way to handle this would be to have a check that the specific error code for a MySQL error resulting from a query buffer violation doesn't try to insert the query, again just putting this out there.
The problem for me was that the query in question was larger than max_allowed_packet would allow, so I just had to increase this a bit. MySQL has some documentation on this setting at the below URL. http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html#sysvar_m... Miriam Natanzon wrote:
I also have this problem.
Any Solution?? It caused me a timeout and then "MySQL server has gone away" response message...
-----Original Message----- From: development-bounces@drupal.org [mailto:development-bounces@drupal.org]On Behalf Of Steven Surowiec Sent: Wednesday, April 22, 2009 7:12 PM To: development@drupal.org Subject: [development] Watchdog dieing on large queries
I've found a workaround for myself for this one but just wanted to put it out there. When a query fails watchdog is used to insert it into the DB, which is fine 99.9% of the time, until that .1% bites you. The problem is that if the query (say inserting a new record into the "cache") causes a query that is larger than the MySQL query buffer limit. This results in a MySQL error that the query is too large to insert, which watchdog then picks up and tries to insert, which fails, which watchdog picks up ... and you get the point. Result? Infinite loop and then either a timeout or PHP runs out of memory and dies. I'd think the proper way to handle this would be to have a check that the specific error code for a MySQL error resulting from a query buffer violation doesn't try to insert the query, again just putting this out there.
participants (5)
-
Iñaki Lopez -
Khalid Baheyeldin -
Miriam Natanzon -
Moshe Weitzman -
Steven Surowiec