Getting started with Test Driven Development - Choosing a Test Harness
Getting started with Test Driven Development - Choosing a Test Harness
TDD is a daunting prospect for the newcomer, but Chris Maiden explains it's not so hard to get going
10th June 2013By chris
Following on from the first blog post in this series, An Introduction to Test Driven Development, Chris introduces the test harness so strap yourself in!
My first experience of work "in the real world" came through work experience. I was lucky enough to get one of only two placements with IBM at Havant and my assigment was to build to specification the wiring loom used to connect the IBM 3745/3746 communications controller to its test harness. These units had to be regularly and thoroughly tested throughout production and so periodically each unit was hooked up to a test harness by means of the aforementioned wiring loom so that a range of tests could be executed against the unit and pass/fail reports and data collated. My time at IBM provided a fabulous insight into engineering/testing and firmly cemented in my mind the concept of a test harness.
In software development, a test harness provides exactly the same functionality, it provides a means of executing tests against software and giving feedback on whether the software is behaving as expected.
The specific test harness you choose will likely vary from project to project and is dependant on a number of factors, the coding language of the project and experience of the developers and test engineers involved may have an impact on the decision making process, a Google search for "test harness for language x" will no doubt turn over a number of options for each language and whilst there’s no hard and fast rule that your test suite and project code language must match, the skillset of the team surely will play a part.
Another major consideration should be around the types of test you want to execute, (unit, integration, system, acceptance etc) as some test harness frameworks lend themselves more towards one or a subset of types. You can of course use more than one test harness if it makes sense for your project with each providing different elements of a test plan strategy: unit tests running on every build and full smoke and functional tests running on release for example.
Consideration also needs to be given to the development methodology being applied - perhaps PHPUnit is a best choice if a purely Test Driven approach is adopted, on the other hand, Behat is a clear winner for Behaviour Driven Development.
I'll leave the last key point for consideration till the end! First, let's briefly review 4 popular test frameworks.
SimpleTest
For those of us working with PHP (and specifically Drupal), there are a number of options. Coupled closely with Drupal is SimpleTest which was moved to core in Drupal 7. It's modeled on the open source SimpleTest unit test framework so those familar with its suite of assertions will be comfortable and will get the extra benefit of helper methods such as drupalLogin(), drupalLogout(), drupalCreateNode() and drupalCreateUser() for establishing fixture[1].
Here's an example of a user login test in SimpleTest taken from the user module and modified slightly for clarity:
<?php /**
* User login test.
*/
class UserLoginTestCase extends DrupalWebTestCase {
public static function getInfo() {
return array(
'name' => 'User login',
'description' => 'Ensure that login works as expected.',
'group' => 'User',
);
}
/**
* Test that a user can login.
*/
function testUserLogin() {
$user = $this->drupalCreateUser(array());
$this->drupalLogin($user);
}
}
As you can see, tests are expressed in PHP code and SimpleTest provides a number of assertions that can be used.
I won't go into much detail here about SimpleTest as many good blog posts already exist about its use and limitations (duration of test run being a major issue which SimpleTest Clone seeks to address, some have had success with switching to MyISAM for tests and others, along with Drupal Quality Assurance have adopted ramdisk with a mixed outcomes it seems.)
Code Enigma offers high performance Drupal hosting with built in support for running automated tests, see our hosting page for more information or get in touch if you'd like to talk it through.
PHPUnit
PHPUnit is a popular open source PHP test framework created and maintained by Sebastian Bergmann which, from its name, is a good choice for unit testing although you can combine it with other libraries, Guzzle and Selenium Server for example enabling browser automation for functional and acceptance testing.
I personally find PHPUnit invaluable and use it on a daily basic, tests are easy to read, write and are quick to run if you're actually doing true unit testing. Of course, that in itself raises a challenge when working with Drupal where dependencies are commonly requested rather than injected. Drupal 8 is heading in the right direction with Mark Sonnabaum leading the introduction of PHPUnit tests in the upcoming release. I recommend watching the video of his recent Drupalcon Portland session on the subject.
A really basic example PHPUnit unit test:
<?php /**
* Adds two numbers and returns the result.
*/
function add($a, $b) {
return $a + $b;
}
class AddTest extends PHPUnit_Framework_TestCase {
public function testadd() {
$this->assertSame(3, add(1, 2));
}
}
Once again, tests are expressed in PHP code with PHPUnit providing perhaps a slightly richer set of assertions than you get with SimpleTest.
Behat
Created by Konstantin Kudryashov, Behat is the PHP Behaviour Driven Development enabler for PHP projects - it enables composition of human readable tests (in Gherkin) that Behat turns into executable acceptance tests. This is the beauty and strength of BDD (and Behat), together they provide a means whereby business owners, developers, project managers and testers can frame requirements in a common language, the language of the business domain and from there derive a suite of acceptance tests that indicate delivered value.
The DrupalExtension module provides additional step definitions for some of the basic operations normally performed on a Drupal site, in addition it provides support for regions defined in the theme layer for more accurate querying of the user interface.
I recently gave a talk about BDD and Behat at the North West Drupal user group - you may find my slides and code helpful.
An example feature and scenario:
Feature: Login
As a visitor
I want login
So that I can manage my account
Scenario: A user enters their credentials and logs in successfully
Given I am on "/"
And I fill in "admin" for "name"
And I fill in "letmein" for "pass"
When I press the "Log in" button
Then I should see "My account"
Expressed in a structured language called Gherkin, Behat along with the Mink extension can take this feature and turn it into an executable acceptance test without the need to write any PHP code.
Codeception
Codeception is a PHP framework created and maintained by Michael Bodnarchuk. It enables acceptable, functional and unit testing which are created by expressing context, actions and expected behaviour in simple expressive PHP code.
At the time of writing, Codeception has switch on module support for a number of PHP projects such as Symfony, Laravel, Yii and ZF but unfortunately no-one has written the equivalent for Drupal yet. Keep an eye on Mike Bell and Paul Byrne who have both written very good blog posts about Codeception, Testing with Codeception and Drupal Projects and More testing with Codeception and Drupal projects
An example Codeception test:
<?php $I = new WebGuy($scenario);
$I->wantTo('ensure that frontpage works');
$I->amOnPage('/');
$I->see('Home');
Tests are composed in simple expressive PHP code.
Conclusion
It's perhaps now apparent that whichever test framework you choose, the end result is much the same: tests are executed either directly against code (unit testing) or against the site itself (functional/acceptance testing through browser automation) and the failures are reported in some way. The key variant these frameworks offer, and in my book, the most important element for consideration is the input. Answering the following pivotal question helps:
How best can tests be expressed for this project and the people involved in its delivery?
Although not explicit, you might want to include the client when you consider the people involved!
1. A fixture is a known application state which tests will run against, establishing a fixture is the work you do to get to that know state, perhaps the creation of a user for example.
Installing 3G USB Modems On LinuxBlog