Shoov - CI tests on the live site
Shoov keeps evolving, and now has an example repo that demonstrates how we're trying to make UI regression simpler, we took some time to implement the second feature we were missing - automatic testing on the live site.
We saw a very strange situation everywhere we looked: Dev teams were writing amazing test coverage. They were going the extra mile to setup a Travis box with environment as close as possible to the live site. They tested every single feature, and added a regression test for every bug. Heck, every commit triggered a test suite that run for an hour before being carefully reviewed and merged.
And then the site goes live - and at best they might add Pingdom monitoring to check it's working. Pingdom at its simplest form sends an http request every minute to your site. If the answer is 200
- it means that all is good in the world. Which is of course wrong.
Our mission is to change this, and bring functional testing to the live site. One that is easy to setup and that integrates with your existing testing and GitHub flow.
The Drupal backend holds the CI build data, including the full log, and status
While Pingdom is wonderful and is alerting us on time whenever a site goes down, its "page is fine, move along" approach doesn't cut it for us. Here are some examples why testing on the production server is a good idea: