[development] Queue as a supplement to cron

[Darrel O'Pry]

On Mon, 2007-03-12 at 18:28 -0700, Neil Drumm wrote:
> On 3/12/07, Boris Mann <boris at bryght.com> wrote:
> > On 3/12/07, Neil Drumm <drumm at delocalizedham.com> wrote:
> > > On 3/12/07, Boris Mann <boris at bryght.com> wrote:
> >
> > > > > Is this functionality something that people would like to see in core?
> > > > > Search module has a 'queue' of unindexed content, would it benefit
> > > > > from having an abstracted queue?
> > > >
> > > > aggregator (and other, contrib-based aggregation) would benefit from
> > > > this as well, I believe.
> > >
> > > Those happen regularly, so cron is best.
> >
> > Errr....except for mass numbers of aggregation where one cron run
> > can't crank through everything. Queue might not be the right solution,
> > but I wanted to point out that current cron architecture is not up to
> > the task.
> 
> Adding individual items to the queue on cron might be a good pattern.
> That is what I'll be doing to import the pile o' XML that changes
> every day, which I've got to deal with for a project.
> 
> I coded this to run queue items on cron until more than half of the
> PHP allowed execution time has been used.


You could even process the queue with a daemon on another server, so you
don't bog your web server. I like it.. I'm already doing it with another
module I've written.

.darrel.