[drupal-devel] Drupal cache statistics

[Jeremy Andrews]

Hi Dries,

> Per configuration we measured more than 150.000 page requests.

Do you know roughly how many pages Drupal generates per minute, on 
average?  Does this fluctuate greatly from minute to minute?  How long did 
it take to get 150,000 page requests?

> If we look at the cache miss percentages of anonymous user requests only 
> (excl. authenticated users) we get:
>
> 1. Strict       : 66%
> 2. Loose, 300ms : 65%
> 3. Loose, 600ms : 64%

I have a feeling something is wrong with the loose cache implementation.

> If we look at the average number of pages served between two flushes 
> (excl. authenticated users) we get:
>
> 1. Strict       : 280 pages/flush
> 2. Loose, 300ms : 182 pages/flush
> 3. Loose, 600ms : 327 pages/flush

You mean 300 and 600 seconds, not milliseconds, right?

Three reasons you might not see any increase in the number of pages served 
from cache even with loose caching enabled come to mind:
  1) you are mis-measuring
  2) Drupal.org does not have a consistent traffic load, serving pages in 
bursts spread apart greater than the timer on the loose cache
  3) There is something wrong with the loose caching implementation that 
was merged into core.

> 1. Loose caching has NO effect on drupal.org's page cache.
> 2. Loose caching has NO well-defined effect on drupal.org's flush ratio.

I'm not in a position to review patches and such at the moment, so these 
may be stupid questions.  But, how are you measuring the flush ratio?  Is 
it based on when a certain function is called, or when we actually query 
the database to mass-delete cache entries?

Do you do any measurements of CPU utilization with the various cache 
configurations?

> Unless we can serve 2000+ pages/flush (random estimate), the cache will 
> never be efficient.
>
> 1. Rework loose caching to flush less frequently.
> 2. Flush partially.
> 3. Maybe there is a bug?
> 4. Remove 'loose caching'?

I'm leaning toward #3 at the moment.

Unfortunately I'm not in a position to look at this for another 7 days. 
When I return to Florida, I'll review what has been merged against what 
I'm using on KernelTrap, and retest it all to verify it's working as I 
originally designed it.  I'll also think about ways to further improve the 
design.

Cheers,
  -Jeremy