Research like you’re wrong: Lessons from user research gone rogue

Jen Lambourne is a Technical Writer at Government Digital Services.

At Write the Docs Portland 2018, she gave a fascinating talk on how the GDS conducts user research for its documentation.

"Technical writers are so close to the product they end up with a cognitive bias,” says Jen. “You forget what it's like to be new and use a product for the first time. You have... THE CURSE OF KNOWLEDGE."

This becomes a cognitive bias and presents unique obstacles for technical writers attempting to do user research. Focusing on your user is so difficult there’s a whole profession around it – technical writers.

And yet technical writers themselves can often lose sight of the user. Bring in user research.

User research is defined as:

“The practice of studying users’ behaviors and motivations to understand more about their needs.” 

This helps technical writers to adopt the cognitive frameworks of users and sweeps away those pesky cognitive biases.

Planning your research

User research is really difficult though, and you shouldn’t attempt it alone. You need a whole team to make the project a success.

“There’s a myth that user research only happens at the start of a project,” says Jen. But continuous research is key to success.

Before your research, “You need to find out what you know and don’t know. And then later find out what you don’t know you don’t know,” says Jen. Identify your assumptions about your docs so you can test these in your research.

It’s important to identify measurable assumptions, such as developers being able to install and configure your program in less than 3.5 minutes.

When planning your user research project, find some people and start small – 5 to 7 people is enough for one round. “5 people will find 85% of all usability issues in documentation,” says Jen.

A small sample can still result in big diversity. This will help reduce the amount of interviewing that you later need to transcribe.

Conducting your research

Once you’ve got everything in place for your research, it’s time to actually conduct the study.

Considering using test prototypes – you don’t need the full application coded and it can even be paper prototypes. “Some of the most successful research I’ve done involves card sorting,” says Jen.

Card sorting is used to evaluate the proposed Information Architecture of a site, or how you propose to lay out the navigation. You can either use physical cards or digital card sorting software like OptimalSort.

Heatmapping is useful for getting customer feedback even using printed out content. This involves asking users to highlight sections of the page in colors to rate their level of confidence in the action they are being asked to perform.

Remember, some users are colour blind, so you need to keep in mind that your tasks are accessible to all users.

In research, stop talking. “Use silence to your advantage,” says Jen. “Your users will fill them with noise. If you feel like you’re talking too much, you probably are.”

“Humans are really polite,” Jen says. That’s why you need to make sure your questions aren’t too leading, as people will want to please you.

And finally, getting consent from your users is vitally important.

Analyzing your research

You now need to feed your research back into your documentation using the specific measurable actions you defined before you started out.

The Hawthorne Effect is a user who is being observed will modify their behavior, so this should moderate how you view your findings.

You can’t really apply quantitative analysis to qualitative analysis, because the numbers just don’t scale.

Don’t underestimate the time it takes to analyse the research as it’s difficult to fit all the analysis in around your day job. Two hours of research roughly equates to one hour of analysis, and doesn’t include presenting to the rest of your team.

Other colleagues will be very interested in what you’ve done, but beware of subjective interpretations. Stick to the facts and record them so you can later do something with your facts.

Answer the hypothesis and present it in a way that your colleagues can appreciate.

Making the most of your research

When conducting your user research, it’s easy to end up with a laundry list of new features. But beware that these can represent user wants – rather than needs.

“Don’t confuse user needs for user wants,” says Jen. “Your users might need a bicycle rather than a lamborghini, although they may want the latter.”

Your job is not to get feature requests to add to your backlog – all you need to do is understand your users and their motivations.

Here’s a link to Jen’s full talk on YouTube.

Images by Kay Smoljak. This work is licensed under a Creative Commons Attribution-Non Commercial-ShareAlike 2.0 Generic License.

Check out our previous blog post on the achievements of the Government Digital Services in service design. 

KnowledgeOwl is our very own knowledge base software, perfect for technical writers of many stripes. Sign up for a free trial of KnowledgeOwl!