Okay, I'll admit it ... PIFR isn't the sexiest, flashiest, or most widely used Drupal module in existence. In fact, if one were to try and identify the number of actual production sites using PIFR today, it'd be a tough job to use up all the fingers on one hand. Counting the number of people who can explain exactly how PIFR performs its magic would likely result in an even smaller total. However, these numbers paint a misleading picture ... though not all of them realize it, hundreds (thousands?) of developers and drupal contributors depend on pieces of that PIFR magic each and every day.

For those of you not familiar with this module, PIFR stands for 'Project Issue File Review' ... and it is the current workhorse behind Drupal's automated testing infrastructure. Every time someone 'git push'es a new commit to the drupal repositories, or posts a patch to an issue queue (whether for Drupal core or one of over 220 other module branches now leveraging the testing infrastructure) ... that commit/patch eventually ends up at a testbot running PIFR. This is where the magic happens. For each new patch or project commit, PIFR creates a brand new Drupal site, downloads and installs the affected module, applys the patch as needed, scans each affected file looking for syntax errors, and then executes a number of simpletest tests to ensure that the patch/commit hasn't accidently broke anything. For a simple module, this might mean verifying a dozen simpletest tests ... for the current version of Drupal 7 core, the number of tests checked is over thirty-two thousand! Once the tests are completed and the results are received, the whole site is deleted and the next patch obtained; triggering the whole process once again ... to be repeated over and over and over with each new patch or commit posted to drupal.org.

I won't go into the process of how a patch in an issue queue actually ends up on a PIFR testbot ... this is a carefully coordinated dance between drupal.org, qa.drupal.org, and the testbot; and a subject for an entire post on its own. But I will mention it ... as one of the new features discussed below eliminates the need for this dance, allowing developers to trigger tests on their own standalone testbot as needed.

Chances are, if this is your first time hearing about PIFR, you can probably skip the rest of this post. However, if you're a developer or module maintainer with a project being tested by PIFR, then read on!

New features in the PIFR 2.6 branch

• Issue #694876: Added sorting and filtering to the 'Branch Tests' page on qa.drupal.org.
  • This should make it easier to find the tests you're looking for, without the need to hunt through an un-sorted list.
• Update PIFR to support 'Coder' module reviews
  • As mentioned above, the test environment's 'Coder' plugin was essentially ignored during the CVS to Git migration. It has now been made functional once again.
• Issue #830838: Allow configurable 'advisory' reviews.
  • Essentially, this allows the test environment to run a series of tests and provide feedback on the results, but disregard those results when calculating whether a patch should be considered a pass or fail.
  • In other words, we should now be able to enable Coder reviews via the testing infrastructure ... and still mark a test as 'passed' even if it fails a coder review.
• Issue #1176982: Added a 'manual test interface' to the PIFR testbots.
  • Not only does this help simplify testing of testbots before they are deployed, developers who host a local testbot (usually on a local virtual machine) can pre-test patches before posting them to an issue queue.
  • If there's enough interest, I'll outline the steps for creating your own local testbot / testing environment in a future post.

Other Bug Fixes

• Issue #1126112: Resolved multiple scenarios which caused a full suite of drupal core tests to be evaluated for contrib module patches.
  • This should significantly reduce the wait time for testing results after a contrib patch is uploaded to an issue queue.
• Issue #1227386: Resolved certain scenarios leading to 'reviewed 0 out of x files', and patches passing with '0 tests passed'
• Issue # 1233248: Added a check and error message for certain '0 tests passed' scenarios
• Numerous other miscellaneous bugfixes & tweaks

What's next?

Here's a sneak peak at some of the current developments within the testing infrastructure, which hopefully can be expected at some point in the near future ...

• Testing of contrib modules with dependencies: Currently, testing of contrib modules which depend on other contrib modules is not supported; but a new project_dependencies module promises to address this shortfall. (Status: Code complete, ready for integration testing.)
• Ability to execute one-time simpletest/coder tests from sandbox/project pages: Initially desired to enable automated coder testing of 'New Project' applications, this feature should also allow for developers to perform on-demand tests during the development stages of a project. (Status: Under initial development)
• Stability and Robustness improvements: PIFR 2.6 helps to address some of the common testbot/testing failure scenarios; I expect to see additional stability improvements and the ability to provide more information regarding the root cause of a failed test in upcoming releases.

Hopefully you find the 2.6 improvements worthwhile ... What's on your feature wishlist for the next PIFR release?