Executing batch cron operations with drupal_queue
Sometimes when developing with Drupal you need to execute heavy scripts during cron, for example: your website has a "subscription" feature where users pay a montly fee to access premium content.
Without getting into implementation details, basicly we need to periodically iterate through every registered user with the subscriber role and check whether his subscription has expired or not and, if it has, remove his access, display a warning, etc.
The Problem
When we are talking about half a dozen users that's really ok, just stick the loop into hook_cron() and we're good! However, in a site with hundreds or thousands of subscribers you will quickly notice that this cron script is really slowing things down and could even take your site offline. It's just too long and too memory-intensive a script to execute on a regular basis with no side-effects.
The Solution
Enter, Drupal Queue.
Drupal Queue is essentially the equivalent of Drupal's well-known Batch API with a twist: its directed towards cron operations!
Pretty awesome right? It's a backport of work done in Drupal 7 and simply put, just works! However, the documentation on the project page is a little scarce to say the least so I'm going to walk you through a standard implementation.
Implementing Drupal Queue
Needless to say, download and install the module as usual. Now, implementing Drupal Queue is as easy as 4 simple steps:
- Implement hook_cron_queue_info()
- Alter your hook_cron() to use Drupal Queue
- Implement your worker callback
- (optional) Add a separate cron process for Drupal Queue workers
So first, let's implement hook_cron_queue_info():
/**
* Implements hook_cron_queue_info().
*/function mymodule_cron_queue_info() {
$queue['mymodule_queue'] = array(
'worker callback' => '_mymodule_queue_worker',
);
return $queue;}
Basically what we do is define one or more queues and define a worker callback for each one. The worker callback is the function that is going to do all the heavy-lifting.
Ok, now let's write our hook_cron() to make use of the queueing functionality:
/**
* Implements hook_cron().
*/function mymodule_cron() {
$items = array();
// this is the list of items we will iterate through
// in the above example if would be a list of all subscribers
$queue = drupal_queue_get('mymodule_queue'); // grab queue
$queue->createQueue(); // initialize
foreach ($items as $item) {
$queue->createItem($item); // each item passed as param to worker
}}
Pretty simple right? Sweet! Now let's implement our worker:
/**
* Worker callback
*
* @see mymodule_cron_queue_info().
*/function _mymodule_queue_worker($item) {
// do whatever you want with $item
// basically you can do all the heavy-lifting you want and not worry about timeouts, memory limits, etc}
So we are pretty much done at this point... except for:
Add a separate cron process for Drupal Queue
By default Drupal Queue is pretty smart and will use subsequent cron runs to work through it's queue. However, we can greatly ease this process by adding a separate cron process specifically for Drupal Queue.
If you are using Drush simply add "drush queue-cron" to your crontab. Otherwise, there's a "drupal_queue_cron.php" in the module's folder that can be copied to your site's root directory and be added to the crontab just like Drupal's own "cron.php" would.
Why this is important: Say your Drupal cron runs once every 8 hours, that means that tasks added to the cron queue by Drupal Queue will sit around for 8 hours waiting until the next cron run to get processed. That's just silly!
Now, what if we configure Drupal Queue's cron process to run every half-hour. If there are no tasks in the queue there is no harm done but if there are, it will quickly and efficiently process them without and side-effects and without having to wait for the next cron run!
Conclusion
If you ever have to iterate through hundreds or thousands of items or execute any other kind of heavy cron operation do yourself a favour and use Drupal Queue!
Tags:
9Dec2011
Language
English
Category:
Reviewed: Yes