I run my Drupal crons with Drush and Jenkins, and have been running into a race condition frequently where it tells me, Attempting to re-run cron while it is already running, and fails to run.

That error is triggered when the cron_semaphore variable is found. It's set when cron starts, and is deleted when cron ends, so if it's still there, cron is presumably still running. Except it wasn't really - the logs show the previous crons ended successfully.

I dug into it a little further: drush vget cron_semaphore brought up the timestamp value of the last cron, like it was still set. But querying the `variables` table directly for cron_semaphore brought up nothing! That tipped me off to the problem - it was caching the variables array for too long.

Searching the issue brought up a bunch of posts acknowledging the issue, and suggesting that people clear their whole cache to fix it. I care about performance on the site in question, so clearing the whole cache every 15 minutes to run cron is not an option.

The underlying solution to the problem is very complex, and the subject of several ongoing Drupal.org threads:

• variable_set() should rebuild the variable cache, not just clear it
• Optimize variable caching
• Concurrency problem with variable caching

Following Drupal core dev policy now (which is foolish IMHO), if this bug is resolved, it has to be resolved first in 8.x (which won't be released for another 2-3 years), then 7.x, then 6.x. So waiting for that to work for my D6 site in production isn't feasible.

As a stopgap, I have Jenkins clear only the 'variables' cache entry before running cron:drush php-eval "cache_clear_all('variables', 'cache');"

That seems to fix the immediate problem of cron not running. It's not ideal, but at least it doesn't clear the entire site cache every 15 minutes.