First mobile usability test of D8

In early October 2012, we performed our first mobile usability test on a prototype toolbar for Drupal 8.

The study aimed to get answers to the following questions:

  • How do users use the toolbar on their mobile devices and on their desktops?
  • Are the users able to easily navigate to the tasks that they want to perform?
  • Do users understand the interaction pattern of the toolbar?
  • How does the experience differ when on mobile and when on desktop?

Overall, the results were encouraging, but also showed that we still have a lot of work ahead of us.

The study was a part of the Mobile initiative and Spark to make Drupal’s administrative theme and toolbar responsive.

Participant using the new navigation toolbar

Dharmesh initiated the usability test as part of the Spark project, we setup a test and conducted a 6-participant moderated usability study on the new toolbar. The design was tested on desktop and iPhones (and on the iPad in some cases).

Summary of findings

Overall, the toolbar prototype tested well on desktop and iPhones.

“This is a big improvement from where we are at [right now]” (P4)

Most of the participants (4 out of 6) had positive reaction. The rest of the participants (2) had a neutral response to the change. The mobile version was significantly better received than the desktop experience. It is also worth noting that participants were mostly neutral about having different mobile vs. desktop versions of the toolbar.

Major Positives:

  • Participants thought that the design was clean, usable, with “nice” icons and was visually appealing.
  • Participants responded very positively to using it on a mobile phone, it was recognizable and easy to navigate.

Major Issues:

  • Collapsing menu items through “>” and the link “Edit shortcuts” are not discoverable (show context).

We have created a more detailed report in the usability group at Toolbar usability study findings.

Drupal 8 is not mobile friendly yet, how can you help out?

Currently, little of what we tested is in Drupal 8 and throughout the test we often ran into Drupal UI that is not mobile friendly; from vertical tabs, secondary tabs, to forms. The mobile initiative has long suffered from a lack of contributors on the UI parts, which we are now slowly catching up on.

Issues#1137920: Redesign toolbar from the ground up so it works on mobile/tablets as well as desktop#1751150: Improve usability of forms on touch screen and small screen devices#1277352: Responsive vertical tabs#1488962: Increase touch target size of admin menu lists

We are looking for contributors with all skills: designers to give feedback on our ideas, sitebuilders to test out patches on various devices and (front-end) developers to make these challenging responsive ideas happen.

If you are would like to be involved:

Mobile testing setup

From a methodology standpoint, it might be interesting to know, that we did a combination of remote testing and on-site testing. During the remote testing we asked participants to turn their webcam front-facing and use their mobile phone in front of it, that allowed us to see their actions without creating too awkward/ unreal situation. We tested participants that matched the following profile:

  • Existing Drupal users who own a smart phone
  • Users who have a need to access/update their website on the phone
  • Haven't tried the new mobile toolbar design

For remote testing, the team used WebEx for screen sharing and recording. For the on-site testing, we used a small €10,- DIY-setup that I created a while ago. It is an acryl form (4mm), that allows the camera to be positioned right above the mobile phone. It can be both attached to the phone or lose.

Moving forward, we intend to do more mobile usability testing on Drupal 8, and make it part of our basic testing effort. It’s exciting to do, as you can learn a whole lot about the scalability of desktop UI patterns.

Co-authored by: Dharmesh Mistry

Original Article: