Hi there! I am considering to write a patch which would split up the accesslog and watchdog tables into a storage table and a smaller table where we'd insert the new events. To still show the end user the combined results, I would use a merge table in mysql (there are other methods for pgsql). The issue is that the merge engine only works for myisam tables so for mysql we'd need to enforce them. Is there anybody who prefers innodb for logging tables and would prefer to not change them to myisam? Cheers, Gerhard
Hi! Gerhard, how would you enforce myisam on shared hosts? Is is reliably available everywhere? Gabor On Sun, 13 Aug 2006, Gerhard Killesreiter wrote:
Hi there!
I am considering to write a patch which would split up the accesslog and watchdog tables into a storage table and a smaller table where we'd insert the new events. To still show the end user the combined results, I would use a merge table in mysql (there are other methods for pgsql). The issue is that the merge engine only works for myisam tables so for mysql we'd need to enforce them. Is there anybody who prefers innodb for logging tables and would prefer to not change them to myisam?
Cheers, Gerhard
Gabor Hojtsy wrote:
Hi!
Gerhard, how would you enforce myisam on shared hosts? Is is reliably available everywhere?
myisam is the standard table engine for mysql, so I guess it would. If the merge engine is available everywhere is a good question, though. Guess you could try the example here: http://mysql.com/doc/refman/5.0/en/merge-storage-engine.html Cheers, Gerhard
What exactly would be the benefit here? My understanding is that InnoDB is already better for write-heavy tables (while MyISAM is read-optimized), so I'm not sure what the goal is. And what would this get us that UNION would not? On Sunday 13 August 2006 12:17, Gerhard Killesreiter wrote:
Hi there!
I am considering to write a patch which would split up the accesslog and watchdog tables into a storage table and a smaller table where we'd insert the new events. To still show the end user the combined results, I would use a merge table in mysql (there are other methods for pgsql). The issue is that the merge engine only works for myisam tables so for mysql we'd need to enforce them. Is there anybody who prefers innodb for logging tables and would prefer to not change them to myisam?
Cheers, Gerhard
-- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson
Larry Garfield wrote: NB: I'd really appreciate it if people on this list could quote messages appropriately and not attach the old message at the bottom.
What exactly would be the benefit here? My understanding is that InnoDB is already better for write-heavy tables (while MyISAM is read-optimized), so I'm not sure what the goal is.
First of all, not everybody uses innodb. We can safely assume that myisam is the standard table type that 98% of all Drupal installs use if they use mysql. We actually used to enforce the myisam table type.
And what would this get us that UNION would not?
A merge table is basically a permanent union (Note: I am not a big database expert, if somebody is, please explain it better). You wouldn't need to bother about knowing how many tables you want to get the data from, etc. My idea is to split our accesslog (and watchdog) into accesslog_storage and accesslog_insert, and add both to a merged accesslog table. During cron runs, the content of the _insert table would be added to the _storage table and the _insert table would be emptied, keeping it nice and small. If now somebody wants to collects a _real_ lot of data this way he'd want to split up the accesslog_storage table further. He could do so and simply add the new tables to the merged table as well and would not need to add the tables to some union statement. Cheers, Gerhard
Gerhard, go for it. Create an enable merge tables page which simply dies if that facility is not available and if merge is not switched on, show only the insert table on the watchdog page. Everyone gets something... Regards NK
On 8/13/06, karoly@negyesi.net <karoly@negyesi.net> wrote:
Gerhard, go for it. Create an enable merge tables page which simply dies if that facility is not available and if merge is not switched on, show only the insert table on the watchdog page. Everyone gets something...
Yep, that sounds like a good solution -- there are then 2 settings -- 1) for the cron'd clean up (acceslog_insert) and 2) for storage. i.e. (1) keep logs for 3 days, (2) keep logs for 3 months. If merge support doesn't work in some way, just the smaller, accesslog_insert is displayed/handled. There was that watchdog hook looking to do flatfiles or something, too? Other means of dumping data could be done by contrib modules as needed. -- Boris Mann Vancouver 778-896-2747 San Francisco 415-367-3595 Skype borismann http://www.bryght.com
karoly@negyesi.net wrote:
Gerhard, go for it. Create an enable merge tables page which simply dies if that facility is not available and if merge is not switched on, show only the insert table on the watchdog page. Everyone gets something...
I prefer a more elegant approach. :p http://drupal.org/node/78503 Cheers, Gerhard
I don't like to impose an infrastructure default that may not be available for most users, nor can they enable it if they want to (think shared hosting where Inno or Merge may not be available). hook_accesslog() and hook_watchdog() is the way to go. We can ship default modules that do what is done today, but contributors can write modules that do syslog, SNMP, pager, monit, nagios, ....etc. The possibilities are endless. I may even volunteer to write a contrib syslog module for both hooks if no one else is working on it.
Khalid B wrote:
I don't like to impose an infrastructure default that may not be available for most users, nor can they enable it if they want to (think shared hosting where Inno or Merge may not be available).
hook_accesslog() and hook_watchdog() is the way to go.
We can ship default modules that do what is done today, but contributors can write modules that do syslog, SNMP, pager, monit, nagios, ....etc. The possibilities are endless.
I may even volunteer to write a contrib syslog module for both hooks if no one else is working on it.
Khalid has the proper solution, I believe. Providing a way for module developers to meet the ever varying hosting infrastructures that people will want to run Drupal on is the best way to solve any problem of this kind.
On 13-Aug-06, at 7:27 PM, Khalid B wrote:
I don't like to impose an infrastructure default that may not be available for most users, nor can they enable it if they want to (think shared hosting where Inno or Merge may not be available).
hook_accesslog() and hook_watchdog() is the way to go.
So...yes, agree. We still have to figure out what to support in core. I guess that means leaving accesslog as is, and having a logarchive module that does as Gerhard has proposed. -- Boris
participants (7)
-
Boris Mann -
Chris Johnson -
Gabor Hojtsy -
Gerhard Killesreiter -
karoly@negyesi.net -
Khalid B -
Larry Garfield