Usability Testing on the Cheap

Think only large companies with big websites (and bigger budgets) can afford to do usability testing? Think again. You may not be able to afford eyetracking software, but putting together a usability study that will point out areas that need improvement isn’t as hard as you might think.

According to Patrick Neeman (, there are four distinct steps to great usability testing on the cheap: writing the test, finding participants, running the tests, and analyzing the results. Each of these can be further broken down by the things you need to keep in mind as you go through them. So let’s take them one at a time.

The first step is to write up the tasks you’re going to test. Neeman recommends coming up with five tasks that you’ll want site users to perform. Don’t make it too easy or too hard; each task should take around six to nine clicks. While you won’t be testing every task with every one of your participants, it helps to have a variety of scenarios.

Notice I used the word “scenario” here. You’re not going to tell your user exactly what to do, since the point is to see if they can figure it out on their own. Say your website provides virtual stores for artists to sell their goods. You might tell your tester that she is a new artist specializing in glassware, and needs to sign on and build a new shop. Or perhaps she just completed several projects, and needs to upload images and post descriptions and prices. Maybe she just completed a commissioned work, and needs to post the images to your site in such a way that only the buyer can see them. I’m sure you get the idea.

By the way, as you’re writing these scenarios, keep in mind that you’re testing the clarity of the website’s language, in addition to the ease or difficulty with which site users perform various tasks. So use terms like “sign in” if your site says “log in” and “search your hard drive” if your site uses “browse.” In other words, use generic terms, not ones that are specific to your site.

After you’ve finished your creative writing, consider the parts of the site you’re testing, and think about coming up with related questions. After all, your users will be interacting with the screens trying to get things done; since you have them there, why not see if they have some specific, helpful ideas? Use leading questions that can’t be answered simply with a “yes” or “no.” I’ll mention the importance of asking questions again later, when I discuss the actual testing.

{mospagebreak title=Choosing Participants}

Very few products and services are intended for “everybody,” so you need to make sure the people you choose for your usability testing represent a good cross section of your site’s targeted users. You’re probably going to have to think outside the usual box when considering the demographics of your site. You need to think carefully to figure out what factors need to vary so you can capture the full range of preferences and abilities.

What do I mean by this? Well, consider Etsy. This website was created for artists and artisans of various types and those who like to buy handmade goods. It lets users sell their wares to an audience as wide as the Internet. You will get everyone from the high school student with a gift for pencil sketches to the dad tinkering in his workshop to the serious artist who makes their living this way full-time to grandmothers just looking to keep their hands busy – and there are a few other categories I’m sure I haven’t listed. And this, mind you, is just looking at it from the seller end. Buyers may be just as diverse, if not more.

Not all of these people will be equally experienced and skilled, either with their own crafts or with technology in general. You know the young emo goth amigurumi maker probably won’t have any trouble setting up a shop and uploading images of her creations – but what about her grandmother, who makes patchwork tote bags and the softest knitted shawls? Using Flickr as his example site, Neeman notes that “It’s best to find users that could be both expert users (a professional photographer) and users that might not have as much experience (an uncle that just bought a camera and uses Facebook sporadically).”

So, I’ve told you to find usability testing participants who would be in your target audience and who represent the range of experience of those who would use your site. Who else should be on your participant list? Believe it or not, you might want to add people who would hate your product. Why? You need the balance. If you’re doing something poorly on your website, you need to make sure that you’ll hear about it – especially if someone else is doing it better. Testers who have no emotional investment in your product won’t back away from telling you when you’re doing it wrong. Neeman notes that “The best suggestions I’ve gotten have been from  users that provide more negative feedback (‘You know, this id done better on this other site’) than constantly good feedback.”

Tomorrow I’ll cover the final two steps: performing the usability test, and evaluating the results. See you then!

[gp-comments width="770" linklove="off" ]