Testing Drupal with WebDriver browser mode vs Headless browser mode

There is not a lot of documentation available about what's the difference between running a browser in WebDriver mode vs Headless so I did some digging...

by
Jibran Ijaz
/ 14 August 2018

Apparently, there are two ways to run Chrome for testing:

  • As WebDriver
  • As Headless

WebDriver:

There are two ways to run Chrome as WebDriver:

Using Selenium:

Run Selenium standalone server in WebDriver mode and pass the path of ChromeDriver bin along with the config e.g. Selenium Dockerfile

This works fine with Nightwatch standard setup, \Drupal\FunctionalJavascriptTests\JavascriptTestBase and also with Drupal core's new \Drupal\FunctionalJavascriptTests\WebDriverTestBase.

Using ChromeDriver:

Run ChromeDriver in WebDriver mode e.g. chromedriver Dockerfile

This works fine with Nightwatch, JTB, and WTB.

Headless:

Using Chrome

Run Chrome browser binary in headless mode. e.g. Chrome headless Dockerfile

Nightwatch is not working with this set up, at least I was unable to configure it. See https://github.com/nightwatchjs/nightwatch/issues/1390 and https://github.com/nightwatchjs/nightwatch/issues/1439 for more info.
\DMore\ChromeDriver can be used to run the javascript tests.

Using ChromeDriver

Using Selenium ChromeDriver can be run in headless mode something like this:

const fs = require('fs');const webdriver = require('selenium-webdriver');const chromedriver = require('chromedriver');const chromeCapabilities = webdriver.Capabilities.chrome();chromeCapabilities.set('chromeOptions', {args: ['--headless']});const driver = new webdriver.Builder()  .forBrowser('chrome')  .withCapabilities(chromeCapabilities)  .build();

DrupalCI is running ChromeDriver without Selenium and testing Nightwatch and WTB on it.

Conclusion

The question is which is the best solution to run Nightwatch and JTB/WTB tests using the same setup?

  • We had seen some memory issues with Selenium containers in the past but we haven't run into any issue recently so I prefer this and you can swap Selenium container to use different browsers for testing.
  • We have also seen some issues while running ChromeDriver in WebDriver mode. It just stops working mid-test runs.
  • I was unable to get Headless Chrome working with Nightwatch but it needs more investigation.
  • Headless ChromeDriver setup on DrupalCI is quite stable. For JTB this would mean that we could use anyone from \Drupal\FunctionalJavascriptTests\DrupalSelenium2Driver and DMore\ChromeDriver.

Please share your ideas and thoughts, thanks!

For more info:

Tagged

Testing, nightwatchjs, Functional Testing

Original Article: