I... think I follow?  It sounds like you want an approach like the search index uses; when you tell it to rebuild the search index it doesn't truncate the index but marks all records as "dirty", so they get reindexed over time by cron.

You may also find this of use:

http://drupal.org/project/views_content_cache

--Larry Garfield

On 10/18/10 8:31 AM, Tomáš Fülöpp (vacilando.org) wrote:

Yes, Larry, I did experiment with that approach as well. It allows
caches to expire at different times, but a) you need to keep some sort
of approximate overview about the various expiration delays you're
setting (so that they don't have much chance happening at the same time)
and b) whenever such cache happens to expire, it still gets deleted at
cron and then will have to be re-calculated during the precious time of
page request.

Currently I am thinking about the following approach - crude pseudo code:

// Make sure no other cache has been started over defined period
* set buffer period between cache refreshes $cbuffer = 15 seconds
* if ( (time()-$lastcacherun) > $cbuffer )

// Check if the cache at hand has expired
* Set some cache life time $clife
* SELECT `created` FROM `cache` where `cid` = 'cachedobjectname'
* if ( (time()-$created) > $clife )

// Refresh the expired cache
* $lastcacherun = time()
* recreate cachedobjectname
* DELETE FROM `cache` WHERE cid = 'cachedobjectname'
* cache cachedobjectname using cache_set  with CACHE_PERMANENT

What do you think?






On Mon, Oct 18, 2010 at 14:34, larry@garfieldtech.com

<mailto:larry@garfieldtech.com> <larry@garfieldtech.com

<mailto:larry@garfieldtech.com>> wrote:

   Just to make sure, have you tried using the minimum cache lifetime
   on the performance page?  It essentially says that a cache record
   will always last at least that long, even if a clear is requested
   for it. That's your first step if you're finding some caches
   clearing too frequently (especially the expensive filter and page
   caches).

   --Larry Garfield


   On 10/18/10 5:22 AM, Tomáš Fülöpp (vacilando.org

   <http://vacilando.org>) wrote:

       Hi,

       In D6, after all caches are cleared, or after a lot of them
       expire and
       get emptied by cron, the server load spikes seriously because
       all such
       caches need to be re-populated.

       Since this happens more and more on sites I work on, I have been
       thinking about using another approach in my modules, in the
       sense that
       caches would be /refreshed/ rather than cleared and
       re-populated. Each
       cache refresh would run depending on e.g. a simple variable
       storing last
       time stamp of any other cache refresh.

       This would assure that a) all cached values would be available
       at all
       times, b) caches would never be re-calculated all at the (near)
       same time.

       I am about to write logic for this, but wanted to first check with
       others in the list -- perhaps some of you know or can point to an
       elegant solution that already exists.

       Thanks!

       vacilando