[development] Batch API on cron

[Alex Barth]

On Jul 15, 2010, at 10:34 AM, Moshe Weitzman wrote:

> drush has some built-in support for redispatching (typically used when
> when memory gets too low). this is used my migrate module and drush's
> own updb command. though others have noted that it can be just as easy
> to bail out when you get too low on memory and call same drush command
> on your own.

That's useful to know.

Let me digress a little:

I have recently seen how Fusion Tables http://tables.googlelabs.com  
does uploads in the background without repeated page loads through the  
client.

I like that. After the file upload, you can leave the upload page and  
come back in an hour to check whether it's done.

Drupal's Batch API tempts us all too much to use client based batching  
which isn't good for very heavy tasks. It's on my long list for  
improvements to Feeds to figure out how to build clean dispatching for  
batched tasks in the background.

Drupal 7's queue and redispatching in drush is going to be of  
tremendous help. I'd appreciate any further pointers.

>
> On Thu, Jul 15, 2010 at 9:04 AM, Ken Rickard  
> <agentrickard at gmail.com> wrote:
>> Drush scripts (especially bulk node processing) are subject to  
>> hitting
>> PHP memory limits when processing large amounts of data.
>>
>> Anyone have ways around that issue?
>>
>> On Thu, Jul 15, 2010 at 8:49 AM, Moshe Weitzman  
>> <weitzman at tejasa.com> wrote:
>>> I think drush scripts are your best bet. CLI PHP is not not  
>>> subject to timeout.
>>>
>>> On Thu, Jul 15, 2010 at 5:01 AM, Sven Decabooter <sdecabooter at gmail.com 
>>> > wrote:
>>>> Hi,
>>>> I'm reading contradicting posts about running Batch API processes  
>>>> on cron.
>>>> This is for Drupal 6 BTW.
>>>> I have tried implementing a batch functionality that should be  
>>>> run on cron,
>>>> but it doesn't seem to process the work that needs to be done.
>>>> I assume this is because running the cron through a commandline  
>>>> command
>>>> doesn't allow for javascript...
>>>> So my questions:
>>>> - Have I implemented Batch API incorrectly, and should it  
>>>> normally work also
>>>> on cron?
>>>> - What is the best way to run a process that would normally  
>>>> trigger a php
>>>> script timeout? Can I use the Queue module for that?
>>>> I'm sure plenty of people have already tried doing this, so I'm  
>>>> not sure why
>>>> I can find little consistent information about it.
>>>> Thanks for your feedback.
>>>> Sven
>>>
>>
>>
>>
>> --
>> Ken Rickard
>> agentrickard at gmail.com
>> http://ken.therickards.com
>>

Alex Barth
http://www.developmentseed.org/blog
tel (202) 250-3633