Whether you only have a prototype or a full-fledged product, it’s highly recommended to run monthly usability tests to make sure that whatever you’re working on is usable and the user experience is excellent.

If you’re wondering what you can do to make your usability tests more structured and organized so that you can learn more from them, this guide is for you. Let’s get started!

First off, always keep the two Golden Rules of Usability Testing in mind:

1. Any testing with anyone is better than no testing (with no one!)
2. A bit of testing earlier is better than a lot of testing later.

In this post, I will introduce you to the kind of light-weight usability testing described in Steve Krug’s books, “Don’t Make Me Think” and “Rocket Surgery Made Easy.” Steve calls this kind of testing “Do-It-Yourself Usability Testing” since it’s supposed to be cheap, easy-to-do and takes just a morning a month.

The straightforward idea behind this is to find a few participants, ask them to come in and go through a list of user flows you want to test, observe the problems they run into, and finally make a list of issues to fix. Sounds simple enough, but very few of us actually do it. The goal of this post is to make you confident enough to run at least one usability test session this month. I ran my first usability test only a year ago, and I must say that it’s actually a lot of fun!

Before we get to the test itself, here are a few things to note:

1. Reserve one morning a month (say the third Thursday every month) for a round of testing, debriefing, and deciding what to fix.
2. Test with three participants each round. Recruit loosely, and grade on a curve — i.e., you don’t need to find someone who fits the exact mold of your ideal user since most usability problems can be uncovered by testing with just about anyone.
3. If you are part of a big company and have the budget, you can recruit via Craigslist and offer a $50 gift card for an hour of the participant’s time. If like the rest of us you don’t have those kind of resources, don’t worry — you can ask your friends, your existing users, or even go to a café and ask strangers for 15 minutes of their time in exchange for buying them a coffee.
4. If you’re doing this as part of a bigger team, get as many observers as possible to observe the tests in a separate observation room — these will be the designers, engineers, project managers, executives, etc. Or, in case of side projects, it’ll be just you later in your room!

What happens during the test?

During a usability test, you will record the participant’s voice, and the screen, as well as share both these streams live with observers in another room. A typical one-hour test can be broken down into:

1. Welcome (4 mins): Explain how the test will work so that the participant will know what to expect.
2. The questions (2 mins): Ask the participant a few questions about themselves. This helps put them at ease and gives you an idea of how computer-savvy they are.
3. The Homepage tour (3 mins): Open the Home page of your site, and ask the participant to look around and tell you what they think of it. This will give you an idea of how easy it is to understand your home page, as well as how familiar the participant is with your domain.
4. The tasks (35 minutes): Watch the participant perform a series of tasks you have prepared for them beforehand. If you’re building a SaaS product and you’re testing out your subscription flow, a typical task could be to find the Pricing page, compare various plans, and Subscribe to one of the plans with a provided test credit card number. Encourage the participant to think out loud as they perform the task (see the video at the end of the post for a sample test.) It’s crucial that you let them work on their own and not ask them any leading questions, or give out any clues or assistance.
5. Probing (5 mins): Ask the participant any questions you may have about anything that happened during the test and about any issues that people in the observation room may have. Also, answer any questions that the participant may have at this point (don’t answer them during the actual tasks since you’re testing how they’ll perform with no one around.)
6. Wrapping Up (5 mins): Thank them for their help, and give them their gift card if you promised one while recruiting them.

The Debriefing

During the breaks between successive tests, ask the observers to write down the top 3 usability problems that they see. During the debriefing, focus ruthlessly on deciding to fix the most severe problems first. Here are a few other recommendations:

• Keep a separate list of low-hanging fruit. These are the problems you can typically fix with one-line code changes, but have a huge impact on task completion rates. Joel Califa calls them “tiny wins”. Here’s an example:

• Resist the impulse to add things — instead, try to tweak your existing design to fix the problem.
• Take “new feature” requests with a grain of salt. Participants will often suggest new features, but when you probe them further, they will admit that they will likely not use the features they are proposing. Instead, try to get to the root of the problem that the participant faced and was trying to fix on their own by suggesting you that new feature.
• Ignore the problems where the user goes astray for a bit but comes back on track by themselves. These are usually not worth investing much time unless you see a pattern across multiple participants.
• Good design is a delicate balance, so when fixing a problem, ensure that you aren’t introducing new ones.

Remote testing and unmoderated user testing

Remote testing is very similar to an in-person usability test, except that the participant is at their home/office and you conduct the testing via screen sharing and voice call.

Unmoderated user testing is another way to test, where you specify your website, the tasks you want the users to do, and get back video recordings of people trying to accomplish those tasks. Usertesting.com is the leader in this space, but note that a single 30-minute test costs about $50.

Resources

You can download checklists, interview script, consent form, a demo video at Steve Krug’s site here: Downloads for Rocket Surgery Made Easy

Conclusion

We will revisit usability testing in The Sprint Process Day 5: Test and Learn. The process recommended there is practically the same with small variations.