estimating execution time left and hook_cron
Hello, I am maintaining the module feedapi imagegrabber, which pulls the images from the original url of the feed items (just like google reader). It is implemented on the top of feedapi and hence implements the feedapi_after_refresh() hook, which is basically fired by the feedapi after it has refreshed the feed. Now, as my module download images from an external website, I can never be sure about the time it will take to complete the job of downloading images for all the feed items. Therefore, sometimes I get a 'fatal execution time exceeded error'. How can I know how much execution time is left, so that I can return on time by saving the rest of the work as a cron job? I have gone through the source code of FeedAPI which implements hook_cron(). It uses a variable called 'cron_semaphore', which I am guessing is some way to share execution time but am unable to get how. I will really appreciate if someone can explain the above to me or provide some tutorial links on writing a good hook_cron implementation. -- Regards, Nitin Kumar Gupta http://publicmind.in/blog/
Nitin, http://us2.php.net/manual/en/function.ini-get.php will let you get the php setting, though there may be better ways of handling this. The cron semaphore is just to make sure more than one instance of the cron isn't running at once (a signaling system). - Ken Winters On Sep 18, 2009, at 1:16 PM, nitin gupta wrote:
Hello,
I am maintaining the module feedapi imagegrabber, which pulls the images from the original url of the feed items (just like google reader). It is implemented on the top of feedapi and hence implements the feedapi_after_refresh() hook, which is basically fired by the feedapi after it has refreshed the feed.
Now, as my module download images from an external website, I can never be sure about the time it will take to complete the job of downloading images for all the feed items. Therefore, sometimes I get a 'fatal execution time exceeded error'. How can I know how much execution time is left, so that I can return on time by saving the rest of the work as a cron job?
I have gone through the source code of FeedAPI which implements hook_cron(). It uses a variable called 'cron_semaphore', which I am guessing is some way to share execution time but am unable to get how. I will really appreciate if someone can explain the above to me or provide some tutorial links on writing a good hook_cron implementation.
-- Regards, Nitin Kumar Gupta http://publicmind.in/blog/
You may want to check out the Job Queue module, as it provides the functionality you are looking for. You could either add a dependency on it (recommended) or at least read through the code to see how it works, to prevent timeouts (It has a feature to check execution time and stop processing if the time reached is half of the max execution time). http://drupal.org/project/job_queue -Mike __________________ Michael Prasuhn 503.512.0822 office mike@mikeyp.net http://mikeyp.net On Sep 18, 2009, at 10:16 AM, nitin gupta wrote:
Hello,
I am maintaining the module feedapi imagegrabber, which pulls the images from the original url of the feed items (just like google reader). It is implemented on the top of feedapi and hence implements the feedapi_after_refresh() hook, which is basically fired by the feedapi after it has refreshed the feed.
Now, as my module download images from an external website, I can never be sure about the time it will take to complete the job of downloading images for all the feed items. Therefore, sometimes I get a 'fatal execution time exceeded error'. How can I know how much execution time is left, so that I can return on time by saving the rest of the work as a cron job?
I have gone through the source code of FeedAPI which implements hook_cron(). It uses a variable called 'cron_semaphore', which I am guessing is some way to share execution time but am unable to get how. I will really appreciate if someone can explain the above to me or provide some tutorial links on writing a good hook_cron implementation.
-- Regards, Nitin Kumar Gupta http://publicmind.in/blog/
hey, thanks for pointing this out but as I see the job queue module uses a timer_read('page') to see the current execution time. Who starts this timer? Is it set by the PHP when the execution starts? becuase if it so, then the work is done. One more thing I would like to ask from the experienced ones, I have been working with PHP for just a while now (say 10 months), but what I am really confused about is how do we get to know so many things that are of our interest? I mean sometimes I just end up reinventing the wheel. -- Regards, Nitin Kumar Gupta http://publicmind.in/blog/ On Sat, Sep 19, 2009 at 1:15 AM, Michael Prasuhn <mike@mikeyp.net> wrote:
You may want to check out the Job Queue module, as it provides the functionality you are looking for. You could either add a dependency on it (recommended) or at least read through the code to see how it works, to prevent timeouts (It has a feature to check execution time and stop processing if the time reached is half of the max execution time).
http://drupal.org/project/job_queue
-Mike __________________ Michael Prasuhn 503.512.0822 office mike@mikeyp.net http://mikeyp.net
On Sep 18, 2009, at 10:16 AM, nitin gupta wrote:
Hello,
I am maintaining the module feedapi imagegrabber, which pulls the images from the original url of the feed items (just like google reader). It is implemented on the top of feedapi and hence implements the feedapi_after_refresh() hook, which is basically fired by the feedapi after it has refreshed the feed.
Now, as my module download images from an external website, I can never be sure about the time it will take to complete the job of downloading images for all the feed items. Therefore, sometimes I get a 'fatal execution time exceeded error'. How can I know how much execution time is left, so that I can return on time by saving the rest of the work as a cron job?
I have gone through the source code of FeedAPI which implements hook_cron(). It uses a variable called 'cron_semaphore', which I am guessing is some way to share execution time but am unable to get how. I will really appreciate if someone can explain the above to me or provide some tutorial links on writing a good hook_cron implementation.
-- Regards, Nitin Kumar Gupta http://publicmind.in/blog/
On Fri, Sep 18, 2009 at 4:03 PM, nitin gupta <nitingupta.iitg@gmail.com>wrote:
hey, thanks for pointing this out but as I see the job queue module uses a timer_read('page') to see the current execution time. Who starts this timer? Is it set by the PHP when the execution starts? becuase if it so, then the work is done.
Core Drupal starts this timer for you in bootstrap.inc.
One more thing I would like to ask from the experienced ones, I have been working with PHP for just a while now (say 10 months), but what I am really confused about is how do we get to know so many things that are of our interest? I mean sometimes I just end up reinventing the wheel.
-- Regards, Nitin Kumar Gupta http://publicmind.in/blog/
On Sat, Sep 19, 2009 at 1:15 AM, Michael Prasuhn <mike@mikeyp.net> wrote:
You may want to check out the Job Queue module, as it provides the functionality you are looking for. You could either add a dependency on it (recommended) or at least read through the code to see how it works, to prevent timeouts (It has a feature to check execution time and stop processing if the time reached is half of the max execution time).
http://drupal.org/project/job_queue
-Mike __________________ Michael Prasuhn 503.512.0822 office mike@mikeyp.net http://mikeyp.net
On Sep 18, 2009, at 10:16 AM, nitin gupta wrote:
Hello,
I am maintaining the module feedapi imagegrabber, which pulls the images from the original url of the feed items (just like google reader). It is implemented on the top of feedapi and hence implements the feedapi_after_refresh() hook, which is basically fired by the feedapi after it has refreshed the feed.
Now, as my module download images from an external website, I can never be sure about the time it will take to complete the job of downloading images for all the feed items. Therefore, sometimes I get a 'fatal execution time exceeded error'. How can I know how much execution time is left, so that I can return on time by saving the rest of the work as a cron job?
I have gone through the source code of FeedAPI which implements hook_cron(). It uses a variable called 'cron_semaphore', which I am guessing is some way to share execution time but am unable to get how. I will really appreciate if someone can explain the above to me or provide some tutorial links on writing a good hook_cron implementation.
-- Regards, Nitin Kumar Gupta http://publicmind.in/blog/
-- Khalid M. Baheyeldin 2bits.com, Inc. http://2bits.com Drupal optimization, development, customization and consulting. Simplicity is prerequisite for reliability. -- Edsger W.Dijkstra Simplicity is the ultimate sophistication. -- Leonardo da Vinci
Ohh, thats great, thanks pals, I think my problem will be solved by this. but if someone still has something better in their heads, please let it out. -- Regards, Nitin Kumar Gupta http://publicmind.in/blog/ On Sat, Sep 19, 2009 at 2:00 AM, Khalid Baheyeldin <kb@2bits.com> wrote:
On Fri, Sep 18, 2009 at 4:03 PM, nitin gupta <nitingupta.iitg@gmail.com>wrote:
hey, thanks for pointing this out but as I see the job queue module uses a timer_read('page') to see the current execution time. Who starts this timer? Is it set by the PHP when the execution starts? becuase if it so, then the work is done.
Core Drupal starts this timer for you in bootstrap.inc.
One more thing I would like to ask from the experienced ones, I have been working with PHP for just a while now (say 10 months), but what I am really confused about is how do we get to know so many things that are of our interest? I mean sometimes I just end up reinventing the wheel.
-- Regards, Nitin Kumar Gupta http://publicmind.in/blog/
On Sat, Sep 19, 2009 at 1:15 AM, Michael Prasuhn <mike@mikeyp.net> wrote:
You may want to check out the Job Queue module, as it provides the functionality you are looking for. You could either add a dependency on it (recommended) or at least read through the code to see how it works, to prevent timeouts (It has a feature to check execution time and stop processing if the time reached is half of the max execution time).
http://drupal.org/project/job_queue
-Mike __________________ Michael Prasuhn 503.512.0822 office mike@mikeyp.net http://mikeyp.net
On Sep 18, 2009, at 10:16 AM, nitin gupta wrote:
Hello,
I am maintaining the module feedapi imagegrabber, which pulls the images from the original url of the feed items (just like google reader). It is implemented on the top of feedapi and hence implements the feedapi_after_refresh() hook, which is basically fired by the feedapi after it has refreshed the feed.
Now, as my module download images from an external website, I can never be sure about the time it will take to complete the job of downloading images for all the feed items. Therefore, sometimes I get a 'fatal execution time exceeded error'. How can I know how much execution time is left, so that I can return on time by saving the rest of the work as a cron job?
I have gone through the source code of FeedAPI which implements hook_cron(). It uses a variable called 'cron_semaphore', which I am guessing is some way to share execution time but am unable to get how. I will really appreciate if someone can explain the above to me or provide some tutorial links on writing a good hook_cron implementation.
-- Regards, Nitin Kumar Gupta http://publicmind.in/blog/
-- Khalid M. Baheyeldin 2bits.com, Inc. http://2bits.com Drupal optimization, development, customization and consulting. Simplicity is prerequisite for reliability. -- Edsger W.Dijkstra Simplicity is the ultimate sophistication. -- Leonardo da Vinci
Interested parties are welcome to review http://drupal.org/node/578676 which makes cron scale.
participants (5)
-
Karoly Negyesi -
Ken Winters -
Khalid Baheyeldin -
Michael Prasuhn -
nitin gupta