All- In our large Savannah project (http://new.savannahnow.com) we have been running some tests with both Jeremy's file-based cache and with object-based caching. Some notes. 1) I like the filecache patch and have privately sent Jeremy some 4.6 code that remove tthe *wildcard delete barrier to its usage. The solution is to create unique folders for each cache type (variable, filter, page, et. al.) and then purge that folder when sending cache_clear_all('filter', TRUE). Happy to share the code with others. 2) I was under some internal pressure to reduce the sheer volume of db queries on a page. Object caching allows that. In production right now, we cache the following objects: $node $user $block $profile In the cases of $node, $profile $user, we intercept the _load() function and check to see if a cached version of the object exists. It it does, we load it and skip entirely the hook_user hook_nodeapi hook_load steps of the code. If it doesn't exist, we load the object normally and then cache the result. In order to make this work, we also have to know when to flush the individual object cache when a user makes a change. We flush on node_edit, block_save and user_edit, for example. (Apologies for not having good benchmarks, but if memory serves, it increased our page load speed about 15-20% and took 60-70% of the load off MySQL). In the case of a user page, it cuts our queries from around 140 to 25 or fewer after the initial page view. Some problems / issues / goals going forward: 1) Definitely a cache API. Many contrib modules (and some 4.6 core) use cache_clear_all() far too loosely. As a result, we flush the cache far too often. 1a) In fact, the only way to clear a cache of any sort is with cache_clear_all($cid) or cache_clear_all($type, $wildcard = TRUE) or cache_clear_all(). We need more than one function in order to be more clear about what the code needs to accomplish. The block.module, for instance, runs cache_clear_all() on 3 occassions. But it never really intends to clear the entire cache, does it? 2) I'm not sure that I have properly identified all the times to clear the object cache. 3) Some of this is made unnecessary by the fastcache or page cache, but an object cache applies to logged in users as well. For that reason, even caching the $user object would be a benefit. 4) I am more than happy to contribute notes and code to getting this into 4.8. The work I've done is all in 4.6.x, but the approach can apply to CVS. -- Ken Rickard agentrickard Message: 5

Date: Wed, 12 Jul 2006 10:21:49 +0200 From: Dries Buytaert <dries.buytaert@gmail.com> Subject: Re: [development] Caching, caching, caching... To: development@drupal.org Message-ID: <DB3DEF7E-FA41-4AD3-A210-C177CA4B2853@gmail.com> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed

On 12 Jul 2006, at 00:01, Gerhard Killesreiter wrote:

...

Now the question is if we want to go down this road. To properly invalidate the cache requires to implement all sorts of hooks (forum module needs _comment and _user for example) which you would not need otherwise. This complicates the code quite a bit.

Personally, I'd not optimize with MySQL's query cache in mind. If the folks at MySQL improve their query cache implementation, our efforts might have been moot. I'd focus my energy on high-level improvements, rather than low-level improvements. That is, I'd focus architectural changes that reduce the number of SQL queries or that simplify complex queries (eg. less joins). For example, the node table and node_comment_statistics table have a one-on-one relation and could potentially be merged. I'm not sure what it would buy us, but it might be worth it.

Maybe we should run the modified devel.module on drupal.org to see what the most expensive queries are? You know, accumulate results over multiple page views. Do we have recent data query data from drupal.org? Let's start by identifying some key bottlenecks.

-- Dries Buytaert :: http://www.buytaert.net/

------------------------------

-- [ Drupal development list | http://lists.drupal.org/ ]

End of development Digest, Vol 43, Issue 38 *******************************************