On 3/12/07, Neil Drumm <drumm@delocalizedham.com> wrote:
I sometimes find myself building a queue in a hook_cron() implementation. I decided to make a separate module for this, job_queue, http://drupal.org/project/job_queue. It provides an API which takes a function name and array of arguments for later execution on cron. The only UI is a list of queued function calls.
We implemented something similar to this for the CivicSpace ASP. In general, when people are looking for queues they tend to use other programming languages like Python that provide daemons as part of the core language. PHP's daemon capabilities are lacking so a queue supplements this missing piece with a cron job working through the queue. This is particular important when you want to put a lot of load but it can be spread out. For example, changing the schema of a thousand sites or re-indexing search tables for all sites in a multi-site configuration. Is this functionality something that people would like to see in core?
Search module has a 'queue' of unindexed content, would it benefit from having an abstracted queue?
I'd recommend the following features. 1) Error states for the queue. If a job fails it will set an error level. This is particularly useful for queue logging. 2) A way to tell if queue jobs are still running. This would need to be set for each server instance empirically. If your queued job takes longer than the cron cycle, then you can get escalation issues. You'll want to know wether your server can process 1, 2, 3, or 10 jobs in the queue simultaneously. 3) A global counter for multi-site configurations. If there is a queue, you probably want it to be aware of the resources being expended on other Drupal sites on the same server. Cheers, Kieran --
Neil Drumm http://delocalizedham.com
-- To strive, to seek, to find, and not to yield.