Get our docs in a row, part 2: Needs assessment results

This post is part two in a multi-post series on our own efforts to "get our docs in a row" to update KnowledgeOwl's internal knowledge base, Silly Moose. For the full context, head to our first post in the series: Get our docs in a row, part 1: Internal needs assessment.

In this post, I share the results of our internal needs assessment and my takeaways from the exercise.

The results

I excluded myself and a couple other exiting owls from the survey. Of the total nine owls I sent the survey to, seven responded. (As I said, we're a small team!)

When I wrote up my summary, I combined the Current Silly Moose usage and half of the Silly Moose experience section together, since they were both an assessment of the current usage and experience of using the knowledge base.

Here is the full unedited summary I drafted to share with the team.

A quick guide to abbreviations:

• KO = KnowledgeOwl
• KB = knowledge base

Current usage & experience

• 5 owls reference Silly Moose multiple times per day
• 1 references it about once a day
• 1 references it a couple times a month

During that interaction, 100% of owls reference documentation within individual teams like Support, Dev, etc. the most often, though it's worth noting that folks also said they weren't always sure WHERE the things they accessed were.

• 100% of owls access information on how to do some kind of task
• 6 of 7 owls reference how something works at KO and/or how to run a meeting

Overwhelmingly, owls use search to find what they need. The runners-up in most frequent paths are "My Favorites" and "Someone sending me a direct link to things."

Ease of finding what you need (scale of 0-100, with 100 being super easy): This question had the least consensus, with values ranging from 24-90, though the majority were above the 50-mark.

Silly Moose is considered Maybe or Somewhat a single source of truth at KO by 6 out of 7 owls.

Conclusion

It does seem like a lot of us are referencing Silly Moose for day-to-day or week-to-week tasks. Search and specific pathways/frequently referenced things seem to be the most dominant pattern (though it's worth noting that this could be a byproduct of our information architecture not making sense, rather than an explicit user preference).

Moving forward

A clear majority (5 out of 7 owls) would like to see us stick with a single comprehensive knowledge base rather than separate/bespoke knowledge bases. The reasons for this largely seemed to be around search/findability and ease of use. People were worried that having multiple KBs would make it even more confusing to know where to look and increase the amount of time it took to find the answers they needed.

When asked how they would split things into separate KBs, the dominant theme seemed to be splitting out the company handbook content/vision/etc. from everything else. Though nearly every respondent who said this offered a caveat that they didn't want to split things out, felt torn, etc.

Conclusion

For now, we stick with one KB to rule them all, but we focus on making the information architecture more intuitive.

Knowledge gaps

The owls were split on whether the info in Silly Moose is up-to-date and trustworthy. Between "Other" responses and the multiple choice, it is clear that folks feel some information is up-to-date. It seems like the things people reference most often are generally up-to-date, but the status of the rest of the documentation is either unknown or clearly mildly outdated.

In terms of content gaps, it seems like the majority of owls cited support-specific documentation, whether that's on our overall workflows (visuals were mentioned here), how to solve individual tickets or issues, etc.

There was a secondary theme of how KO works/KO history, including things like: "people reference previous decisions or workflows during Braintrust but there's no documentation on those things in SM and our hiring process."

Not a theme, but I adored this suggestion: "Would be nice to have best practices on how to document things internally, especially once we figure out how we want to structure everything."

Conclusion

It seems like what we have is helping folks, but they want moooooooooooore. This is where we'll need to dig into the docs strategy a bit to figure out the best places for new content creation and what those will look like.

Authorship

Authorship in our respondents was a wide and fairly even spread from rarely (once a quarter) all the way up to habitually (1+ times a week). The top motivation for owls to update documentation was for accuracy (100% of respondents), then strengthening our culture (71.4%), and then a tie between sense-making and don't forget.

The most discouraging factor for folks not authoring content was not having enough time to write or update documentation (71.4%). Organization confusion/not knowing where to put things/not wanting to ask was in second place.

Conclusion

Finding time to create or update documentation is a key theme here. It seems like we all value having good, accurate information to work with, but we struggle to find the time to contribute. The confusing organization/information architecture is potentially an easier problem to solve, and one which has been an explicit piece of this quarterly goal from the beginning!

This speaks to the need for some sort of larger process or commitment to good internal knowledge management: whether that's designated docs work time (docs days/docs hours), a process to flag inaccurate/outdated content and for someone to actually update it, or setting up individual owls as content owners for certain sections or pieces of knowledge so they can handle updating it as an explicit part of accountowlbilities (the KnowledgeOwl version of "accountabilities").

Reflection and next steps

The survey responses did surprise me a bit. My key takeaways from the results were:

• People generally seemed a lot more positive about Silly Moose content than our previous team conversations had led me to believe. This was good news!
• There was very little interest in splitting things into separate knowledge bases, which simplified our decisions and work in the upcoming audit.
• My hypothesis was wrong: though lack of formal structure probably is hindering people from being really comfortable authoring content, the big issue seems to be more one of time constraints. Given the size of our team, that's not a shock. In the past, I've organized an annual "Docs Day" to get people working on content; these results indicated that a quarterly or monthly Docs Day might be a good path forward for us to handle updating content.

From this, I drafted a plan for next steps. We'll likely share that plan as well as how it's going in a future post.

But there were other takeaways here that went beyond the survey:

• It wasn't asking too much. I had worried that asking my fellow owls to complete this survey would be seen as burdensome "extra" work. Thanks to the conversations we'd had about this being a company goal and Marybeth's very vocal and ongoing support, we had good participation. A couple owls did have to leave the survey and come back to it later when they had more time, so I likely should have done more prep work estimating how much time it would take/how involved it was when I shared it. But all in all, it went well.
• My off-the-cuff survey design worked okay. I'm not a survey designer (I'm clearly way too wordy!), and I'm sure there were flaws in how I wrote my questions. But the team seemed to appreciate that it was a single consolidated survey, that there were lots of images of Linus, and that it seemed so comprehensive and thoughtful. I was thankful that I asked why and added free-text responses: a lot of the owls took the opportunity to offer far-ranging text comments when given the opportunity. There were some fantastic nuggets in that feedback which may inform our internal docs strategy moving forward.
• We value knowledge management. This one might seem silly to say, given what KnowledgeOwl is, but when I started here in 2018, we had almost no internal documented knowledge at all. The fact that we have a problem of too much/disorganized content is a GREAT problem: it means people do see that having a knowledge base allows them to work smarter/faster, and that some owls have prioritized documenting knowledge because they see value in it. This is a fabulous place to be, and I'm not sure the magnitude of this cultural change really hit me until this survey.
• I should trust my own instincts more. I've been at KnowledgeOwl since October of 2018, and I've been living with Long Covid since I first got sick with Covid in March of 2020. The cognitive issues I've battled as part of Long Covid often make me question my own ability to make good decisions for the good of the team, and so my form of leadership tends to involve a lot of collaboration and asking questions. This was one of the few projects where I just decided what felt appropriate (the needs assessment) and pushed it forward. I likely should do this more. 😅

Looking ahead

Our next post in this series will share the plan I originally created and why that's not what we ended up doing.

Until then, I hope your knowledge is organized and any audits you're doing are going well. If you want to talk best practices or just need moral support through any of that, you can always reach out! I'm much better at solving other people's knowledge management woes than my own, it seems. 🙃