Usability Testing

Test little but test often. Usability testing should be added as a design story in the project backlog. Actually, it should be multiple stories! Because the idea is to test little but test often, adding multiple stories in the backlog for usability testing will help the team prioritize and plan accordingly. When usability testing is a story, it will make our tasks visible to the Agile team during the sprint. It will also help reporting on our progress during scrum and is a great way to internally socialize usability testing.

The SIMPLE Experience

When we’re working in sprints, we need results quickly

In an agile sprint where time is short, we need to get to the analysis quickly. Rapidly and economically launching a user experience test is only part of the equation. Reviewing and analyzing the results, and decision-making in a timely manner is the other half. If we launched a test with 30-minute sessions of 10 participants, we have 5 hours of video to watch, analyze, make annotations, and pull insights so as to enable product and UI optimization. 100 participants would imply 50 hours. This time adds up quickly. We use a testing platform that enable us to quickly review participant sessions, leverage searchable, time stamped & hyperlinked audio transcriptions to locate the most interesting actions and comments, add & share hyperlinked and shareable annotations, and create & share highlight reels

Resist the urge to help quickly

Product developers may eagerly want to help users to use and like their products. But that would defeat the purpose of the usability testing. Our usability test moderators follow the best practice below.

We remain neutral – we are there to listen and watch. If the participant asks a question, we reply with “What do you think?” or “I am interested in what you would do.” We do not jump in and help participants immediately and do not lead the participant. If the participant gives up and asks for help, we must decide whether to end the scenario, give a hint, or give more substantial help.

If you are not measuring, you are not managing

Learning from how our users engage with our product is one way to ensure a positive customer experience. It’s also important, however, to see how our product compares to competitors through benchmarking. Benchmarking our prototype designs against each other, against existing production assets, and against our competition will allow us to identify additional opportunity areas where we can improve our usability design to create a superior customer experience. The best way to do this is using pre-formatted system usability scale questions with automatic calculations of the resulting score. We use comparison metrics such as net promoter score, time on task, and success/failure, which allow us to quantitatively measure our usability and user experience against different design iterations and against the competition, or best practice websites and apps.