Getting Started Guide retrospective: The good, the bad, the ugly

When I began working at KnowledgeOwl in October 2018, one of the first things I wanted to do was to update our Getting Started Guide. I tried to use it as an orientation document for myself and had all kinds of ideas for updating it. It took me until this month (April of 2022) to get a new guide launched. Here's a bit about why we made the change and how that process went.

Why the change?

Our original Getting Started Guide (hereafter referred to fondly as GSG) was very focused on quick, bite-sized instructions to launch a functioning new knowledge base. I appreciated its brevity, but it was also missing some elements that I thought were pretty essential to onboarding new users (such as the difference between the "app" side of KnowledgeOwl and the actual knowledge base, the flexibility of the category hierarchy you can create and the different types of categories, different access and login options, and so on).

It also was a fairly well-constructed document, and I didn't think I could shoehorn small improvements in along the way and have the rest of it still make sense.

Scratching at the back of my mind was also the idea of something that wasn't just transactional or step-by-step, something that might provide a bigger framework to set new users up for success by helping them figure out why they wanted a knowledge base, what "success" might look like, and to plant the seed of good long-term maintenance practices.

In short, I felt we needed a total rewrite of the GSG.

What took so long?

I'm sure the long and winding tale is one that's familiar to you: first, I wanted to be sure I understood the product well enough to construct a good guide. About the time I had the depth of knowledge and competence to tackle the GSG properly, I was juggling so many other things that it just kept getting put off.

And there were plenty of reasons to put it off:

1. Pressure. I knew how important it was to try to get it right, and I didn't want to rush it.
2. Too much planning. I kept trying to get more input from more people. I asked every new hire we brought on for feedback. I researched other companies' approaches. I brainstormed potential layouts and asked our team about pain points they thought the new GSG could address. I don't know if I'd call this analysis paralysis but it was definitely some over-thinking and over-researching.
3. Not easy to incrementally improve. The direction I wanted to take the GSG was very different in style, layout, and content from our existing GSG, so there wasn't a good way to chip away at it. I felt I needed to create a complete package as my first deliverable.
4. Other things kept being louder, squeakier wheels. Twice, I dedicated time to work on it, and both times I ended up having to tackle other more urgent projects instead.
5. Onboarding documentation doesn't have the same feedback loops as other documentation. Your current customer base doesn't really use it, so you're not generally getting the stream of feedback you might have for constantly-referenced feature documentation. Plus, it's a lot harder to identify when it's not working well. While some people who evaluate or trial your product will reach out if they're confused by something in your onboarding materials, others just may disappear. It's hard to tell if that was a poor product fit, a less-than-serious evaluation, or a subpar onboarding experience. That kind of feedback doesn't generally show up in surveys the way, say, missing features do.

But keeping all of that in mind: in hindsight, it seems strange that I put off the GSG for so long. Yes, it was a large undertaking and all of the previous reasons were valid. But there were tons of reasons I should have prioritized it earlier, too:

1. Quality first experience. The GSG is generally a new user's first experience of our documentation, so it should have been one of the first things I updated.
2. Cascading effects. Since it left out some key concepts, its continued existence in that state meant slightly higher conversation loads for our support owls. While those tend to be great touch points for us to have with new users, I'd prefer not to have that barrier to entry. It's also harder to make up for this later if existing customers lack a consistent foundation of conceptual knowledge.
3. "Marketing" function. We as a company don't have any salespeople and generally market by way of referrals and trying to be awesome. As such, the GSG is technically one of the strongest pieces of "sales" documentation we had. Or should have had.
4. Getting users vs. keeping users. Focusing time on the rest of the knowledge base first meant that I was at least subtly prioritizing keeping existing users happy instead of welcoming new users. While we pride ourselves on being customer-driven in our development and documentation, I wasn't striking a good balance between the needs of brand-new users and our existing users, who were often more comfortable reaching out with questions (and more vocal about documentation that seemed off).
5. Data-informed decisions. Our analytics and tracking didn't lie--for visits to the support knowledge base home page, the majority of clicks were on the link to our GSG. So it was being heavily referenced, even just as an exploratory "what's in here" kind of click.

Designing the new GSG: Research

Although I think I spent too much time doing research and planning, many of the tactics I used could be useful for you.

Here are some of the research and brainstorming steps I took, and a quick assessment of how well they worked for me:

1. Captured my own new-user experience when I first started working here to identify potential gaps
   Helpfulness rating: Low
   I came to the existing GSG a bit tainted, since I was a customer and a KO user before I ever started working here. This is certainly where my motivation for the project came from, but my perspective was already a bit skewed.
2. Got feedback from new support owls' experience when they first started
   Helpfulness rating: Low
   This feedback confirmed my own suspicions of individual pain points and gave me specific suggestions for those individual pain points, but rarely helped me get at the bigger picture of the overall design
3. Spoke with some of our senior members to identify pain points they wished our onboarding documentation could offset
   Helpfulness rating: High
   The most helpful thing about these conversations was that I realized that we ultimately needed multiple guides for different entrance paths to KnowledgeOwl. I focused on the totally new to KO/trying to create my first KB pathway that our existing GSG was focused on, but these conversations have given me several other guides I hope to work on in the future. They also offered me alternate perspectives on where new users struggled.
4. Researched the onboarding guides of other products I like and use
   Helpfulness rating: High
   This was hands-down the most useful thing to break my writer's block. I looked at products whose general documentation and marketing presence I enjoyed and evaluated how they set up their guides. (Slack and Asana were some examples I used.) I compared the difference in structure, identified things I liked and didn't like, and from there began roughing out a skeleton for our own GSG. This was key in breaking my research cycle and getting me going.
5. Got feedback from other owls after initial drafting
   Helpfulness rating: Medium
   This step is where I floundered for a while. I felt (and still do feel, to some extent) that the new GSG has some tricky flaws, and I was hoping to get some detailed constructive feedback on the guide before I launched it. But due to some turnover here and a number of large concurrent projects, I waited too long for feedback that mostly didn't materialize. I was able to get specific feedback on certain sections and other elements that was useful, but ultimately I ended up publishing without that level of feedback, and I could have published much earlier if I had made the decision earlier.

Items 3 and 4 were really where the meat of the good research lay. The work there allowed me to create a rough skeleton of the structure I thought I wanted for this initial GSG and to create a backlog of other items/supplementary materials I could include in future iterations.

Research takeaways

The big takeaway from my internal research was that we really needed multiple getting started guides to handle different onboarding pathways for users in different scenarios.

From my external research, the two GSG models I evaluated the most closely were Slack and Asana. Both have changed their format since my original research, but here was my summary of their offerings at the time:

Slack's Getting Started included sections on:

• Intro to Slack: what is it, its features, understand your actions in Slack, navigation, create guidelines for channel names, Slack Guide for Hipchat users, Slack glossary, tour the Slack app
• Explore plans & features: Slack plans & features, partner promotion program, benefits of a paid plan, enterprise grid, message/file/app limits, try paid for free
• Create a new workspace: create a Slack workspace, getting started for workspace creators, onboard your company to Slack, email template for introducing Slack
• Join an existing workspace: join, getting started for guests, getting started for new members...
• Understand roles & permissions: roles in Slack, permissions on a workspace, find members in directory, understand the primary owner role, owners & admins, multi-channel and single-channel guests
• Download the Slack app

Asana's materials are organized into a Quick Start Guide:

• Data structure: project/task structure, navigation, teams, portfolios, etc.
• Create your first project: how to create an Asana project (templates, etc.)
• Organize your first project: sections/columns, sort/filter
• Tips for project planning a.k.a. Turn your project into a collaborative plan: create tasks for all the steps to achieve your plan, tips for creating a great task, Timeline
• Tips for project management a.k.a. Collaborate and manage projects more effectively: Tips for task collaboration/functionality (add task followers, comments, @mentions, etc.) and tips for features usage (timeline, status updates, portfolios)
• Stay updated on work: My Tasks and Inbox
• More learning resources: Links to more detailed tutorials, etc.
  

There were elements of each of these I really liked. Slack's materials seemed to do a good job of anticipating different onboarding pathways--from people who need to create a new workspace to those joining an existing--while also providing general introductory info around the app's basic functionality and where it gets complex.

Asana's materials had a nice blend of onboarding content on "here's how you do your first x..." and a lot of "tips" on best ways to use the tool for certain scenarios. Since I'd been toying with the idea of including best practice content, these tips helped me feel like best practices weren't outside the genre of GSGs.

I tried to combine the best elements of these two approaches. My initial outline looked like this:

• Intro to KO: what is it, quick summary of major features/functionality, basic publishing workflow
• Getting started with a new account/Create a new KB: basically the existing GSG, revamped
• Something around access control/readers/users: distinction between readers vs. users, user permissions and roles, etc.
• Integrating KO with other tools: your website/app, Zendesk, etc.
• Getting started for new admins/Working on an existing KB: Were you just granted admin access to an existing KB?
• How to work in?
  Authoring/publishing workflows
  Audits/update workflows
  Tips on KB layout
  Admin roles/permissions
  Sharing your KB content (Widget 2.0, Zendesk integration, subscription notifications)
  Tracking usage (Reporting?)

Ultimately, I moved access control/readers/users and the integrating KO with other tools as subsections in the new GSG, as I also did with a lot of the "How to work in?" topics, many of which led to Best Practices documentation, as you'll see below!

How it started

The original GSG was a single long page with very specific steps, titled things like Before you begin, Set up your account, Create your knowledge base, Customize your branding, and Set up who can access your site and how.

Each section had perhaps 2-3 paragraphs in it, all quite short. There was minimal linking to other resources. It flowed in a very linear fashion. I liked the succinctness and the cohesive feel of it, but setting up a knowledge base isn't necessarily linear and people can approach it in different ways. Here's how it looked:

Our former Getting Started Guide (GSG)

How it's going

The new GSG is split into a few different sections:

• Intro to KnowledgeOwl: high-level intro to how KO works, reusing some existing GSG content and some written new
• Get started with a new knowledge base: basically the replacement for the old GSG, but in the form of checklists with individual items linking to far more detailed references. Most of this was totally new.
• Evaluate features: All-new content describing the full KnowledgeOwl feature set

Here's what that "Get started with a new knowledge base" now looks like:

The new "Get started with a new knowledge base" guide

Each of those hyperlinked resources opens a new page with a lot more info...so while the landing page seems shorter and less intimidating, there's exponentially more content associated with this guide.

Good choice...or bad?

Where we're going from here

This list seems to get longer by the day, but in no particular order, here are things I'd still like to work on:

1. Explainer videos. I have mixed feelings on video content broadly, since it's a lot harder to maintain than straight-up documentation. But a lot of the Intro to KO or high-level discussion of various features would really benefit from some brief explainer videos, which could also add a slightly more personal touch to the onboarding experience. If I end up going down this path, I'll be posting a reflection on how that went.   
2. Additional GSG pathways. As mentioned earlier, I realized during this research that we really need multiple guides for different scenarios. The big two that spring to mind are transitioning to KnowledgeOwl from another tool and starting as a new admin for an existing knowledge base (which is definitely a trickier endeavor than starting with a blank slate!).
3. Iterate on this guide. I'm concerned that this new GSG is too detailed and overwhelming for a true new user, that maybe I've swung too far the opposite direction. Now that it's out in the wild, I'm going to return to my research, evaluate those decisions with fresh eyes, and figure out if there are better ways to present this material that are less overwhelming.
4. More best practices. I took a somewhat huge scope risk in creating the new GSG by adding the "best practice" templates and guidelines. We've never had any content of this type in our knowledge base, so I was kind of making up our style and approach as I went. I don't necessarily love that this is tucked into a GSG right now, and long-term I'd love to have these best practices surfaced in a top-level category all their own and have more robust offerings. But that touches on information architecture and all kinds of things, so...it's a whole other can of worms.
5. Should we have actual courses for onboarding? I don't know the answer to this question, but if you've made it this far, I'd love to know what you think. I like how some software platforms give you guided video or text "courses" to walk you through things in a more in-depth way, and I do wonder if taking that approach would solve some of my GSG detail angst. But would people actually use these? Is KnowledgeOwl the kind of product whose users would want to watch videos? Do I want to also maintain that content...?
6. Or maybe onboarding webinars? As an alternative to recorded content, I've also considered offering a once or twice a month webinar for new users who could drop in, get a quick walkthrough, and then ask questions. This feels like less of a lift than courses, but potentially a great touch-point to figure out what content should stay in, be added to, or removed from the GSG.

If you've explored the new GSG or even just read this far through the post, I'd love to know your thoughts, feelings, or random observations about any of it!