Drupal memory consumption (was: coding standard question)
On 8/28/07, Jeff Eaton <jeff@viapositiva.net> wrote:
Also, I would like the bikeshed to be blue.
Pink! How about we look at memory usage by Drupal bootstrap and modules? Here is a start http://2bits.com/articles/measuring-memory-consumption-by-drupal-bootstrap-a... Patch your install and start playing ...
On 28 Aug 2007, at 14:13, Khalid Baheyeldin wrote:
On 8/28/07, Jeff Eaton <jeff@viapositiva.net> wrote: Also, I would like the bikeshed to be blue.
Pink!
How about we look at memory usage by Drupal bootstrap and modules?
Here is a start http://2bits.com/articles/measuring-memory-consumption-by-drupal- bootstrap-and-modules.html
So what is the executive summary (i.e. percentage reduction in Drupal 6 compared to Drupal 5)? ;) -- Dries Buytaert :: http://www.buytaert.net/
My reading of the document is, "The basic bootstrap is a bit bulkier, but modules are lighter." For pure core it's a bit of a wash, but I'm betting that as more modules take advantage of the ability to split code between different .inc files (admin.inc, etc), we'll see contrib getting lighter, too. --Jeff On Aug 28, 2007, at 3:58 PM, Dries Buytaert wrote:
So what is the executive summary (i.e. percentage reduction in Drupal 6 compared to Drupal 5)? ;)
It started as a "where does all that memory come from?" for a site that is really bloated and caused Apache to eat 100MB per process, then it evolved into: let us see how much memory old vs new Drupal takes and for what? Eaton is write, that is the summary: Bootstrap is a bit more, but modules are lighter. Here is to hoping that contrib authors of large modules will bother splitting their modules. However, this is not the full story: there is still a lot of memory for arrays and such that happens when you sit on a lot of data, or when you visit certain pages. This patch does not account for all that, and measures only raw module memory usage upon loading. So, Dries, do you want that memory measurement part in core? So devel can use it? This will help people who are porting modules assess whether a split is warranted or not, and measure how much a split helps too. On 8/29/07, Jeff Eaton <jeff@viapositiva.net> wrote:
My reading of the document is, "The basic bootstrap is a bit bulkier, but modules are lighter." For pure core it's a bit of a wash, but I'm betting that as more modules take advantage of the ability to split code between different .inc files (admin.inc, etc), we'll see contrib getting lighter, too.
--Jeff
On Aug 28, 2007, at 3:58 PM, Dries Buytaert wrote:
So what is the executive summary (i.e. percentage reduction in Drupal 6 compared to Drupal 5)? ;)
-- 2bits.com http://2bits.com Drupal development, customization and consulting.
Jeff Eaton wrote:
My reading of the document is, "The basic bootstrap is a bit bulkier, but modules are lighter." For pure core it's a bit of a wash, but I'm betting that as more modules take advantage of the ability to split code between different .inc files (admin.inc, etc), we'll see contrib getting lighter, too.
There are quite a few steps in core we can take to reduce how much memory we use on bootstrap, at least from code. I'm a bit surprised that bootstrap has worsened, but I think the small size of the site is confusing things. One of the places that Drupal really sucks memory is in that big honkin' menu cache. And that cache is gone. That's part of bootstrap. In Khalid's report, the total bootstrap memory for the bloated site was 10,975,408 bytes vs 5,319,968 bytes for the D6.x bootstrap. I suspect we won't see that number go up nearly as much. With luck, contrib will be lighter. Also, the splitting hasn't been completed, nor is it as completely efficient as it could be. In fact, due to the perceived difficulty of some of my initial proposals, we we ended up with is a really watered down version. We still have lots of hooks that are always being loaded, and sometimes I think it's been difficult to tell what APIs are used and what could go into include files. I think further analysis could really move things out. Also, there are parts of core that are always loaded that probably don't need to be. We have a lot of APIs that could be well-served by analysis and if we think some APIs are lesser used, we could require a 'wakeup' type function to run in order to initialize said API prior to use. I'm a lot more interested in the time it takes to bootstrap Drupal than I really am in how much memory we use, though. Loading code is one of the biggest timesinks in Drupal right now, and I think in Drupal 7 we'll want to try to move even further into the realm of making code optional. There is no reason that very simple pages can't run in 30ms, except that we generally spend 70ms (even on fast machines) loading code (let alone actually generating a page).
On 8/30/07, Earl Miles <merlin@logrus.com> wrote:
I'm a lot more interested in the time it takes to bootstrap Drupal than I really am in how much memory we use, though. Loading code is one of the biggest timesinks in Drupal right now, and I think in Drupal 7 we'll want to try to move even further into the realm of making code optional. There is no reason that very simple pages can't run in 30ms, except that we generally spend 70ms (even on fast machines) loading code (let alone actually generating a page).
Loading less code is always a good thing, but on a good dedicated server with op-code caches, it is a diminishing returns situation, and the database becomes the bottleneck rather than code. Here are examples from a typical page load, for an authenticated user (hence no caching). The queries come from MySQL's query cache, and hence the short time. But, the entire page including the blocks, ...etc. takes ~60ms. Executed *121* queries in *22.04* milliseconds. Page execution time was * 82.45* ms. Executed *120* queries in *26.54* milliseconds. Page execution time was * 85.98* ms. I am not invalidating your point: shared hosts need the code optimization. Not only for speed, but memory footprint per process. But for dedicated? Far less so. Back to memory: can people try to replicate these results on both bloated sites and simpler ones? Also, how do we make memory measurement patchless (part in core, need Dries' blessing on this, and part in Devel). -- 2bits.com http://2bits.com Drupal development, customization and consulting.
participants (4)
-
Dries Buytaert -
Earl Miles -
Jeff Eaton -
Khalid Baheyeldin