STC Summit – Using evidence to influence decision making by Karen Schriver

Karen Schriver is an Information Design Consultant and research expert, and she delivered an extremely valuable talk at STC Summit 2019 called “Improving your professional value: Using evidence to influence decision making”.

It’s easy to become overwhelmed by the sheer amount of information out there. People constantly make claims that sound very convincing, but are they really based on truth? Sometimes, trusting our gut is not as useful as taking a very solid look at the evidence.

Karen discussed the value of dissecting and questioning arguments, and techniques you can use to become a more discerning consumer of information.

Assessing the evidence

Being able to assess the evidence is very important for technical writers. We come across mountains of evidence all the time in our jobs, and claims that people make using certain facts. People have strong opinions on exactly what constitutes effective technical writing, and we have to be able to argue our own case.

But first, what is an argument? An argument is made up of a) a claim, and b) a rationale.

A claim is a conclusion that a speaker is trying to convince you to believe. A rationale is the practical explanation they give for why you should believe their claim.

For example, one claim is: Madcap Flare is a technical writer’s favorite tool. The rationale was a survey conducted, where the majority of participants stated Flare was their favorite tool.

Forming a critique

An argument can then be met with a critique. Do we believe this claim, or disbelieve it, and why? In this first example, the statement was based on a survey of people attending MadWorld – a conference sponsored by MadCap Software. We could critique this claim based on the fact that this was a biased sample, since most people attending the conference would be likely to favor MadCap Flare.

A critique is the evaluation of a claim using the supporting evidence. For example, when evaluating a research study, be aware that the conclusions the researchers draw can be subjective. They may exaggerate the evidence, cherry pick the evidence, or use a non-representative sample of participants.

Be suspicious of language like “always”, and make sure you ask “what is the data?”. Don’t take any claims and rationales at face value.

Qualifying claims

As we mentioned earlier, claims need to be qualified. Ask yourself, what is the rationale for this claim? Where does it come from? And then look at the data. Remember that claims can often be overgeneralized from the data.

As another example, we can evaluate the claim: images are always better than text at communicating information. 

This is a popular opinion, and yet there are several factors that come into play. Images are not necessarily a better choice for communication if users don’t understand the context in which the image is used.

Karen used the anecdote of picking up what she thought was an innocent pack of tissues on a business trip to Japan. She was unaware that the image on the packet was advertising the services of a prostitute when she offered the tissues to her business associates. Needless to say, this was an embarrassing mistake.

Karen aptly demonstrated with this example how a lack of awareness of context – in this case, an inability to read Japanese and to understand cultural norms expressed within the image – meant that she misunderstood what was being communicated.

Another example: Writing in informal language makes a text easier to read. 

Sometimes, writing in informal language – eg, using slang, contractions – does make a text easier to read. But usually only for native speakers of english.

Contractions do make comprehension easier for native english speakers – up to a point. Experience matters when reading english, and some types of (negative) contractions actually cause some native speakers to make more errors.

While positive contractions are effective, negative contractions are not. In fact, many visual and linguistic features affect comprehension, and we cannot consider either type in isolation.

Critiquing the evidence

Karen gave a fantastic talk on how to evaluate arguments. One personal tip I find very useful is to think about who would benefit if people were to believe this claim? I would be more suspicious of study conducted by a group with a vested interest in the outcome. And especially if there is a financial benefit involved.

In effect, don’t be bamboozled by the research. Ultimately, you must try to become a “critical consumer of evidence” and argue from evidence rather than opinion. This is great advice for anyone, but in particular for technical writers.

Main image: Karen Schriver (center)