I don't want to hijack the file cache maintainer thread, so I am starting my own thread about this since I have some questions. I did talk to Robert about this in January, and I have done extensive testing with his code. His code patches core files all over the place in order to check memcache if there is a cached version available and then load that first. He has stated at http://drupal.org/project/memcache that "A set of patches that can be applied selectively to Drupal to cache various objects for high performance. The patches are necessary and this cannot be avoided." However, if we are doing object level caching for nodes than we can cache a complete loaded node without the need to hit the database until a node_save, until a new module is enabled or until the max life is hit. If the code in node_load hit the static variable $nodes first, then secondly hit cache_get then we would not need to have a set of patches, and this could be an option in the performance settings. You could then allow the pluggable cache.inc that I just realized has been merged into HEAD (As of January) to handle storage. This would only take a small change to the core modules in a few places. For example in node.module, if (is_numeric($param)) { if ($cachable && isset($nodes[$param])) { return is_object($nodes[$param]) ? drupal_clone($nodes[$param]) : $nodes[$param]; } $cond = 'n.nid = %d'; $arguments[] = $param; } could be changed into something like this: if (is_numeric($param)) { if($cachable) { if (isset($nodes[$param])) { return is_object($nodes[$param]) ? drupal_clone($nodes[$param]) : $nodes[$param]; } //Attempt to load from cache return false on cache miss $node = cache_get($param, 'cache_node'); if($node) { //Load into static $nodes $nodes[$param] = is_object($node) ? drupal_clone($node) : $node; return $node; } } $cond = 'n.nid = %d'; $arguments[] = $param; } I have time to work on this and offer performance testing. I am just not sure how far anyone has gone down these roads or is currently working on it. I have been working on custom module development for clients for about 7 months now and I have not really had time to contribute to the Drupal project personally other than by spreading the good word and having clients use Drupal. This is one area that I feel could be substantially improved upon in order for us to allow Drupal to scale beyond the current ability. According to Dries testing on his website, he has seen Drupal into the 80+ Requests per second area (ref. http://buytaert.net/drupal-webserver-configurations-compared). With some of the testing I have done with caching, I have seen it go as high as 500-700 Requests per second. In my testing out of the box I have seen Drupal do about 10 requests per second, with normal caching about 100 requests per second and with aggressive caching as high as 250 request per second. Thanks, Steve Rude On 3/6/07, Khalid Baheyeldin <kb@2bits.com> wrote:
See this http://drupal.org/project/memcache
Talk to Robert about it before you duplicate it.
On 3/6/07, Steve Rude <steve@achieveinternet.com> wrote:
I have done much testing with memory based object caching using memcached and I think it would be nice if there was the ability to choose what type of caching you wanted (memory/file/db) and have the cache.incfunctionality basically an abstraction layer that the storage mechanism can be taken care of by other include files ( e.g. cache-memcache.inc cache-file.inc. The same way that the database abstraction works.
I would be willing to begin some work on this if anyone is interested. We are building a Drupal site now that will need to scale to >30m page views per month and I would like to work within an accepted method to help grow and scale Drupal.
If anyone has any prior work or projects that can be looked at to help get this going down the correct path, please point me in the right direction.
Thanks,
Steve Rude
On 3/6/07, Karoly Negyesi <karoly@negyesi.net> wrote:
Hi,
I might be mistaken but I thought file caching was something much sought after. I found posts relating to it as early as 2002.
And yet, when the possibility has arisen for it to happen cleanly (pluggable cache.inc), the code for it has been dumped into CVS on August 26, 2006 and not a single cvs commit since then.
It's not used much according to the issues but that probably has something to do with not having a single release.
Is the community truly interested? If yes, then please a maintainer step up.
And yes, I have contacted the current maintainers three months ago to no avail.
Regards,
NK
--
Steve Rude Lead Web Developer
800-618-8777phone / 858-225-0479 fax www.achieveinternet.com steve@achieveinternet.com
Achieve Internet is a Division of Web Page Maintenance, Inc.
-- 2bits.com http://2bits.com Drupal development, customization and consulting.
-- Steve Rude Lead Web Developer 800-618-8777phone / 858-225-0479 fax www.achieveinternet.com steve@achieveinternet.com Achieve Internet is a Division of Web Page Maintenance, Inc.