We are pleased to announce that PIFT 6.x-2.7 has been released, and is now deployed on Drupal.org. PIFT stands for 'project issue file test', and is the d.o component of Drupal's automated testing infrastructure. This latest release introduces a couple of exciting new features for project maintainers and patch writers, and you may notice some changes to your patch status messages over the next few days as a result ... read on to learn more about the first of these changes and what it means to you.

Current Patch/Branch under Test added to qa.d.o Status Page

As demonstrated in the attached screenshot, the testbot status page now indicates what patch is currently being tested for each testbot.

Test Results now available on Drupal.org
New Testing Tab

As of this morning, git vetted users should notice a new 'Automated Testing' tab on each project page. Selecting this tab opens a new page, displaying the PIFT 'branch test' results for that project, if available.

If a project does not have any branch test results available, users can trigger the testing of a new branch using the 'Test Additional Branches' fieldset. This will trigger a one-time test of the selected branch. Note that this behavior is different than the 'enable testing' checkbox for a project (which enables recurring testing of that project's branches and patches).

Once triggered, the status of that test will be set to 'queued'. This means that d.o is prepared to send the test request to the testing infrastructure, but the request has not actually been sent yet.

After a couple of minutes, the testing infrastructure should pick up the test, and d.o changes the test status to 'sent'.

Typically, testing is only required for development branches (i.e. branches ending in '.x'), and on projects containing simpletest tests. Performing Branch tests on projects which do not contain simpletest tests is somewhat redundant; so I'd suggest refraining from doing so, if you can.

Once the testing infrastructure has completed the testing, it returns the test status to d.o, which updates the table appropriately. Clicking the 'View Test' link will take you to the testing results page for that test on qa.drupal.org.

When Testing Fails

The most common failure scenario is when the testing completes successfully; but one of the individual tests fail. Clicking the 'view test' link on d.o takes you to the results page for that test on qa.drupal.org. If you scroll down the page, you should find a 'Non-pass' section in the test results, which identifies each of the test groups which contain failed tests. Expanding the group will display the actual test filename, line number, and failure message; which should help you trace down the source of the issue.

Occasionally, a test will file with slightly more obscure messages, such as "Unable to initialize database", "Failed during run-tests.sh", or "Unable to initialize checkout directory". Typically, these messages indicate temporary issues with a particular testbot. When these are encountered, the suggested approach is to attempt a 're-test' of the branch to try and reproduce the error ... and if reproducible, create an issue so that the error can be investigated by the testing infrastructure maintainers.

As there is often confusion as to which queue an issue should be posted (PIFR? PIFT? Infrastructure?), we've created a new project issue queue to deal strictly with testbot-related issues. The new queue, found at http://drupal.org/project/testbot, will serve as a one-stop shop for testing related issues ... and we will handle migrating issues from this queue to the actual projects involved as needed.

Known Issues

Occasionally, the testing infrastructure will complete a test, but the result gets lost while being communicated back to the d.o servers. This is due to an existing (but previously undiscovered) bug in the system, which is made much more apparent with the introduction of the Automated Testing tab. When this occurs, a test will appear to remain in the 'sent' state for long periods of time ... but clicking through to the test results on qa.d.o will show the test as completed (with a valid Pass/Fail result).

While the test result from qa.d.o provides you with the actual result, it is not reflected on d.o ... and because d.o thinks the test is still running, it does not allow for the test to be re-queued through the automated testing tab interface. In order to resolve this situation, the recommended solution is to remove the stuck test from the automated testing tab display (using the 'remove test' link), and then queue that branch for re-testing using the 'Test Additional Branches' form in the collapsible fieldset.

Next Steps

Over the next few days, we'll begin to roll out the 'Advisory Coder Reviews', which will be the subject of my next post.