[development] Batch API on cron

[Khalid Baheyeldin]

I ran into this before, and implemented a workaround.

No amount of unset() would free memory.

The workaround was to ration the number of loop iterations to do via the
script, and then end it and start a new one from where you left off.

while true
do
  #php CLI or drush script goes here
  NUM=$?
  TOTAL=`expr $TOTAL + $NUM`
  if [ $NUM = 0 ]; then
    break
  fi
  sleep 30
done

The PHP CLI or drush script returns the number of nodes processed. If it is
0 then the script exits. If there are more, then another iteration is done.

This way, PHP's memory does not grow indefinitely and then bombs out
eventually.

On Thu, Jul 15, 2010 at 9:04 AM, Ken Rickard <agentrickard at gmail.com> wrote:

> Drush scripts (especially bulk node processing) are subject to hitting
> PHP memory limits when processing large amounts of data.
>
> Anyone have ways around that issue?
>
> On Thu, Jul 15, 2010 at 8:49 AM, Moshe Weitzman <weitzman at tejasa.com>
> wrote:
> > I think drush scripts are your best bet. CLI PHP is not not subject to
> timeout.
> >
> > On Thu, Jul 15, 2010 at 5:01 AM, Sven Decabooter <sdecabooter at gmail.com>
> wrote:
> >> Hi,
> >> I'm reading contradicting posts about running Batch API processes on
> cron.
> >> This is for Drupal 6 BTW.
> >> I have tried implementing a batch functionality that should be run on
> cron,
> >> but it doesn't seem to process the work that needs to be done.
> >> I assume this is because running the cron through a commandline command
> >> doesn't allow for javascript...
> >> So my questions:
> >> - Have I implemented Batch API incorrectly, and should it normally work
> also
> >> on cron?
> >> - What is the best way to run a process that would normally trigger a
> php
> >> script timeout? Can I use the Queue module for that?
> >> I'm sure plenty of people have already tried doing this, so I'm not sure
> why
> >> I can find little consistent information about it.
> >> Thanks for your feedback.
> >> Sven
> >
>
>
>
> --
> Ken Rickard
> agentrickard at gmail.com
> http://ken.therickards.com
>



-- 
Khalid M. Baheyeldin
2bits.com, Inc.
http://2bits.com
Drupal optimization, development, customization and consulting.
Simplicity is prerequisite for reliability. --  Edsger W.Dijkstra
Simplicity is the ultimate sophistication. --   Leonardo da Vinci
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.drupal.org/pipermail/development/attachments/20100715/9351f462/attachment.html