tech writing on This is important

Developing an AI strategy for documentation

Thu, 20 Nov 2025 10:44:05 +0000

If you find yourself worried about being left behind because you don’t know what to do about AI and technical writing, this blog post is for you.

Lots of people are using AI. As Anil Dash puts it in his blog post I know you don’t want them to want AI, but…:

using AI tools is an incredibly mainstream experience now.

Even if you’re not personally using AI, many readers of the documentation you write likely are, which means your documentation needs to meet them where they are. You need an AI strategy for your documentation.

Why you might need an AI strategy

Information discovery is changing. When you’re looking for answers to a question you have, you probably do a search on Google. It’s been years since Google launched, so it feels like second nature.

Before search engines existed, people looked up information in other ways — asking librarians, reading encyclopedias, visiting a trusted website (shoutout Encyclopedia Britannica). After search engines launched, people started to look at the search engine results first (thanks, Ask Jeeves). And if the results were useful and relevant (and they often were), people started to trust the results presented by search engines and rely on them when seeking information.

Someone looking for information about your software product can review results containing your documentation, but also third-party resources, and YouTube videos:

However, with the launch and popularity of ChatGPT, Claude, and other AI-based tools, habits are shifting again for many people. People are experimenting with different ways to seek information. One of those ways is using AI to learn about things:

Over the past few months, I’ve been talking with folks about how they find information and learn about a product. ChatGPT and Claude and Gemini (or at least Google’s “AI overview”) are consistently mentioned as one of the main methods people use. Even if the output isn’t that reliable, visiting an AI tool is still a key part of the information-finding flow.

AI tools are used for all sorts of information-finding flows:

Answering basic questions about how to accomplish a task.
Decipher the potential root cause of an error message and provide next steps for resolution.
Provide solutions to intricate application design questions.
Suggest syntax for code, a formula, or a SQL statement.

You need an AI strategy for your documentation to enable customers using AI to be more successful when using your product to accomplish their goals.

Define your AI strategy

The components of an effective AI strategy include the following:

Develop: Partner with teams building AI into your product.
Discover: Evaluate and improve your content and content strategy.
Measure: Measure how people use AI to interact with your documentation.
Adapt: Explore use cases for AI in your documentation practice.

Partner with teams building AI into your product

If you partner with teams building AI functionality into your product, you can enrich those product-relevant AI tools with documentation context.

Two common modalities for in-product AI tools are chatbots and agents:

AI chatbots explaining tasks in the product.
AI agents performing tasks in the product Oftentimes, agents have some sort of explanatory component. For example, an AI agent that rewrites documentation markup to fix a syntax error is functioning as an agent, but it might perform tasks as part of a conversational interface like a chatbot. .

For both chatbots and agents, documentation is a critical component to ensuring the success of the functionality. Why? Because documentation provides the authoritative source of instructions for performing tasks successfully with the product.

If an AI agent is provided with instructions for performing tasks, it can be more effective in completing those tasks, which makes the person trying to use the AI agent more efficient and successful at their goal.

For an AI chatbot, an ideal experience can look like what Casey Smith describes in 10x Impact: Inside Payabli’s Documentation Revolution:

Customers can ask questions like “build me a config for this service,” and the chat generates working configurations from our documentation.

The process of providing in-app documentation is a lot less complex when one interface can provide on-demand assistance. Instead of a clunky experience where you’re configuring elaborate mappings between product pages and documentation pages, or attempting to make webpage-sized content coherent in a 100px square popover.

If your customers have an existing pattern of completing tasks that your company is already planning to add AI into, make sure you’re involved so that you can add documentation context.

Hot take

You can invest in building a chatbot with a tool like Fin, Inkeep, or Kapa. But I don’t recommend placing the chatbot on the documentation site.

When you build your own chatbot and provide it on your docs, you create uncertainty for your customers and readers:

Is the response going to contain accurate and trustworthy information?
Is this chatbot going to be more reliable than ChatGPT or Claude?

Documentation is perceived as an authoritative source of information. If the official documentation provides a way to get inaccurate or misleading information, readers might lose trust in the documentation itself, and ask: Are AI chatbots docs? (No.)

When customers get stuck with your product, they often have to leave to get help. The advantage of an AI chatbot is that it’s a simple way to provide (qualified, yet potentially unreliable) assistance with using the product inside the product. People expect a chatbot within the product to have context of the authoritative how-to documentation because it exists within the product itself.

Instead of adding yet another interface or entry point for customers to learn, discover, and interact with, focus on providing assistance with the product inside the product.

As part of your AI strategy, identify if there is a team building AI functionality into your product, and then partner with them. If there isn’t, partner with someone in engineering to explore options or opportunities for adding an in-app documentation chatbot that can provide that on-demand assistance.

Improve your content and content strategy

Writing high-quality content for humans also helps AI-based tools, but there are still some accommodations that you can make in your content to help AI tools make better sense of it:

For other guidance on writing for LLMs (and humans too), see the following resources:

The blog posts listed in Optimizing docs for LLMs in the November 2025 Write the Docs newsletter.
Writing documentation for AI: best practices by Kapa.ai, a tool that provides retrieval-augmented generation (RAG)-based chatbots.
Optimizing content for Fin by Intercom, who provide a RAG-based chatbot called Fin.

Expose your content to LLMs

LLMs perform markedly better at answering questions about how your product might work when they have access to the context available on your documentation site.

To help improve the likelihood of high-quality answers from LLMs, implement the following practices:

Set up an LLMs.txt file, essentially a sitemap that points to the raw markdown files of your content, which is easier for LLMs to process.
Expose a model context protocol (MCP) server for your documentation site to enable AI agents to search and fetch results from your documentation, or even perform interactive tasks in your API documentation.
- For more about what MCP is and how it might help, see The Model Context Protocol (MCP) and its impact on technical documentation from Cherryleaf.
- For even more details, check out the podcast episode MCP servers and the role tech writers can play in shaping AI capabilities and outcomes with Tom Johnson, Fabrizio Ferri Beneditti, and Anandi Knuppel.

Provide decision support and user-centric content

Customers use your product to accomplish goals they have, like “keep track of my work”. Customers aren’t thinking “today I am going to use a Trello board and cards”.

If you write feature-focused documentation, you need to make sure your content ties together the features in coherent, user-centric content that addresses customer goals.

For example:

Feature-focused	User-centric
Create a card on a Trello board	Track tasks on a Trello board

Or you can split the difference: Create cards to track tasks on a Trello board.

Your customers might already be asking for this type of content (or silently wishing for it), but the LLM tools also need user-centric content to help answer questions about your product more accurately.

When people look for help, a product feature might be the answer, but it’s not the question. People ask chatbots questions, so your documentation must address the question too. Don’t write FAQs (really, don’t). Instead, write user-centric content that addresses both the question and the answer!

If your content only includes information about creating cards, the LLM might make something up because the context about the user goal (tracking tasks) is missing from the documentation. A human might do a decent job at guessing based on prior mental models of task-tracking software, but an LLM needs more help to make that association Trello isn’t a great example because products that are available directly to consumers have somewhat less of a risk with this because there is a substantial amount of third-party content also being created that might fill that gap for your documentation. .

To alleviate the risk of hallucination, write user-centric content that includes decision support to help address questions like “How can I track tasks in Trello?” without literally writing content titled “How can I track tasks in Trello?”.

Write precise and clear content

When LLMs process content from a webpage, the content is split into pieces. For a RAG-based chatbot, this processing is referred to as chunking, and LLMs like ChatGPT and Claude can also chunk content to make retrieving context from linked sources more efficient.

Due to this information processing method, you might want to invest more effort on writing precisely (and possibly also update your style guide!). As quoted in the Write the Docs November 2025 newsletter on Optimizing docs for LLMs:

according to a recent Intercom article by Beth-Ann Sher (Optimizing content for Fin) writing “as if you’re doing a radio interview and you don’t want to be quoted out of context” can be more effective than FAQs.

Given that guidance, it’s crucial to avoid language shortcuts like the following:

Unclear antecedents
Excessive use of relative pronouns
Positional or relational language like “below” or “earlier”

Instead, make references explicit. Refer to specific steps by number, titles of tables, or restate the subject instead of using a relative pronoun. Your writing might get more verbose, but it will also get a lot more clear.

For example, given a section of content like:

Before you eat dinner, set up a table for the cheese platter. This helps ensure that people can access the cheese at any time. It should include the following:

Cheddar

Emmentaler

Gouda

For guidance selecting an appropriately sized platter, refer to the steps above.

You might instead rewrite the section of content to read differently:

Before you eat dinner, set up a table for the cheese platter to ensure that anyone can access the cheese at any time. The cheese table should include the following types of cheese:

Cheddar

Emmentaler

Gouda

For guidance selecting an appropriately sized cheese platter, refer to step 2 of this hosting guide, “Select your serving dishes”.

I’ve exaggerated these examples, but as writers, we often take language shortcuts like using relative pronouns or unclear antecedents. By taking the long way and making these references explicit, not only can LLMs more easily process the content, but so can people that process information differently.

For more details about these writing improvements, I recommend the guidance to Be precise in the Splunk Style Guide (which, for the record, predates LLMs).

Measure how people use AI to interact with your documentation

An important component of your AI strategy is understanding how many people are using AI to learn more about your product and interact with your documentation. For the most part, you can measure it!

In addition to SEO (search engine optimization), web marketing has started to refer to AEO (answer engine optimization) and GEO (generative engine optimization) as new acronyms to care about for measuring and optimizing content for LLMs and AI tools.

The tactics for SEO vs AEO or GEO are similar, but measuring AEO and GEO requires a bit more effort than measuring SEO.

As part of measuring AEO and GEO, I suggest expanding your web analytics monitoring to measure the following for your documentation:

Track referrer traffic originating from LLM chatbots, like chatgpt.com or claude.ai.

Don’t consider referrer traffic a complete representation, however, because tracking only referrers is inherently incomplete due to limitations with how different sites and browsers handle referrer traffic.
Identify AI-attributable traffic in the user agent strings, such as traffic from AI crawling bots or attributable to an MCP server enabled for your documentation.

For a list of known user agents, see Major AI Bot User-Agents on llmsCentral, or AI user-agents, bots, and crawlers to watch from Momentic Marketing.

If you have server-level website monitoring, you can also track more data in request headers, looking for the Signature-Agent field in traffic for your site to identify traffic from AI agents. Read more in Simon Willison’s post ChatGPT agent’s user-agent.

Evaluate the performance of AI tools when answering questions

Evaluating the performance of AI tools for answering questions is also something you can do as part of an AI strategy for documentation This level of evaluation is extremely time-consuming and resource-intensive, so if you’re only planning on implementing a retrieval-augmented-generation (RAG)-based documentation chatbot, evaluating LLM performance at this scale is likely excessive. . However, this is another case where I’d recommend partnering with any teams working internally on AI functionality to ensure that you aren’t duplicating efforts.

When evaluating the performance of AI tools, you can pay for large-scale QA through tools like Profound or Amplitude, or perform and track evaluation suites that you develop with tools like Gauge or Langfuse.

If you perform manual QA, devise a set of high-value questions to evaluate, choosing from the questions that customers most frequently ask support. If that data isn’t available to you, consider the following sources:

Common information that customers search for, guided by particularly long queries typed into search.
High-stakes questions that are important to get right about how your product works.
Comments that people leave in page quality feedback responses for the documentation site.

After you identify the set of questions, define a high quality “ground truth” answer for each question to measure the performance of the LLM-based tools against.

After you have the questions and the answers, ask the questions, record the responses, and score the accuracy of the results against the ground truth answer.

Performing evaluations like this over time helps you monitor the quality of results that customers encounter when looking for information.

If you plan to make changes to your content to help LLMs, perform these evaluations before making those changes to develop a wobbly baseline Performance and quality of LLM-based tools is somewhat inconsistent due to the changes in model performance, system prompts in use, and other factors. Therefore, a wobbly baseline is all that I think you can expect. for your product and documentation performance.

Then, after making changes, repeat the evaluation after some time has passed. I recommend waiting at least a week, or even up to a month, to test changes in external systems. With any luck, the external AI tools will be more successful at providing more reliable or higher-quality responses for the questions being asked when compared to your wobbly baseline.

Consider testing your documentation directly

If you specifically want to evaluate how well your documentation is optimized for an LLM, you can try asking the AI tool directly, as Casey Smith suggests in AI chat as user research when your reader is a bot:

Running content through an AI chat to ask what it thinks is one of the greatest uses for AI as a technical writer. I’m thinking of it as user research, only my user is an LLM with a chat interface. With the adoption of AI chatbots, it makes sense to include them in our content testing and review plans, right?

The important distinction here is that you’re not replacing user testing with a chatbot, but instead testing the success of a different kind of documentation user — the LLM-based chatbot. This sort of testing is one way to evaluate how effectively an LLM-based tool can retrieve relevant content from specific pages of your documentation when directly prompted to do so.

Explore use cases for AI in your documentation practice

Many teams at software companies are being required to use AI. If you’re not yet in that situation, you might be wondering how long you have until you’re asked about using AI in your work.

If and when your leadership team asks you what your plans are for AI content creation, you want to be prepared. Compile a list of what you might use AI to accomplish in your work.

A list of use cases makes it clear that you’re thinking about AI, and that you’re thinking beyond the most basic consideration for using LLMs in technical documentation:

  ---
config:
  theme: 'base'
  themeVariables:
    primaryColor: '#a32fab9f'
    lineColor: '#a32fab9f'
---
graph LR
  A[AI can generate content] --> B[Your job is to generate content]
  B --> C[AI can do your job]

As Fabrizio Ferri Benedetti breaks down in his excellent post Code wikis are documentation theater as a service, generating content from code is not an effective use case for AI in technical writing, so think beyond that.

You want to be strategic and consider where AI might add value to your documentation process. What are the chores and tasks that you don’t like to perform, or that you haven’t found time to do because it would require too much expertise that you don’t have at the moment?

For each use case on your list, consider how you can test the output and how you will keep a human in the loop to ensure quality output. How do you know that the AI is performing better than a person might? Can you measure the effect of using the tool on your productivity?

Some example use cases that might fit on your list of AI use cases for technical documentation:

Extend your static site generator to include a custom theme for admonitions using CSS written by an AI assistant.
Draft a template for new documentation pages to simplify contributions.
Develop style guide linting rules in Vale based on your current style guide.
Use an AI-based code reviewer like CodeRabbit to review PRs for style and valid syntax.
Write alt text for images that humans can review for usefulness and accuracy.
Convert a screenshot of a diagram into a mermaid diagram.
Generate sample data for an interactive tutorial.
Write a script to auto-generate reference content from code.
Analyze the metadata of your documentation pages to build a knowledge graph.

The Write the Docs newsletter from July 2025 includes some additional use cases. See How documentarians use AI (or LLMs).

Leadership might not know the intricacies of your job, but you can demonstrate both your ~ growth mindset ~ and your expertise by learning about the tools available to you and proposing ways to use them Spoilers: You don’t have to actually use them! .

Implement your AI strategy

AI is changing the way people discover and learn. As a result, to grow and learn as a technical writer, you need to know how to deliver content to those people.

Developing an effective AI strategy for your documentation workflow doesn’t mean adopting everything I’ve described here. Pick and choose what makes sense for your organization, personal bandwidth, or team maturity.

Partner with teams building AI agents and chatbots in your product
Make your content easily available to AI chatbots and agents
Deliver conceptual decision support content
Write precise and clear content
Measure AI-driven web traffic
Evaluate LLM performance for specific product questions
Explore AI use cases for your own workflows

No matter how tempting it might be, try not to ignore the existence of AI entirely. Defining your own AI strategy is both a proactive and defensive endeavor — learn about a new technology, and defend your expertise with that knowledge.

Save a horse, use a content strategy

Thu, 13 Nov 2025 18:30:13 +0000

You can lead a horse to water, but you can’t make it drink. In this metaphor, technical documentation is the water.

You can spend a lot of time making the pond look nice and accessible. Provide clear, clean water, some lily pads, a water feature to make it easier to drink. You invest in the structure of the pond to make sure the water doesn’t get stagnant, and even add a nice sign to indicate “fresh water here”.

Credit: Simon Carey / Horse Pond / CC BY-SA 2.0

But if the route to the pond is treacherous — difficult to follow, unclear, full of pitfalls — then the horses won’t visit your beautifully curated pond. You won’t be able to lead the horse to the pond with the fresh, high-quality water. Instead, the horses will drink out of a muddy puddle — because the puddle is right in front of them where they were going already.

I’m stretching this metaphor to its limits to underscore the importance of having a content strategy. It’s not enough to write effective, accurate, and useful content. You also need to identify how people discover your content — invest in documentation discovery and evangelism.

If people don’t know how to get to your content, if it doesn’t appear in their pre-existing paths, they won’t discover it. And if they don’t know what’s in it or what the quality of your content is, they won’t use it.

Consider the context

When you write documentation, you must consider the context in which it exists:

How do customers learn about your product?
How do they discover helpful content?
What do they do when they have a question?
Where do they go for help when they get stuck?

If your documentation isn’t available and accessible where your customers need it, it almost doesn’t matter what you write because it won’t get used.

Instead, customers will turn to alternatives:

Google search AI overview
Chat support with live humans
LLM-based chat interactions like ChatGPT or Claude
Training from third-party consultants
Blog posts published on your company website
Videos created by external developers

These content types can be useful, but many of them are expensive to provide (and access) and are frequently not well maintained. Resources that are only available to some folks or are outdated are not as useful, reliable, or authoritative as official documentation. And if customers are using these content types instead of rather than in addition to the official documentation, you have a problem.

Improve access and awareness

So what can you do?

Ask people what they do when they get stuck. If they don’t mention the documentation, ask them why — identify the hurdles hindering access to the pond.
Make the documentation easy to get to. Pave the way to the beautiful pond you’ve curated by adding entry points like links from the website and the product and improving SEO.
Evangelize the documentation. Tell people about it, talk about the great resources available internally to the folks that interact directly with customers.
Shift priorities to focus on enhancing discovery instead of writing feature-driven content. Accept that the pond can grow some algae as long as the water remains potable.

You might discover when you talk to customers and internal stakeholders that the documentation you thought was excellent isn’t actually fitting the need — maybe the pond is too shallow to provide effective hydration to the horses, or the shoreline is difficult to access, or only some horses can find their way to the pond.

Getting feedback can help you prioritize your work accordingly to help ensure that your documentation gets used. If your technical content isn’t available where your users look for it, your content won’t get used — no matter how good it is.

Humans are creatures of convenience. Build your beautiful pond, but don’t forget to make it easy to access.

Credit: Sunny Ripert / Chevaux au bord de l'eau / CC BY-SA 2.0

Are AI chatbots docs?

Wed, 09 Jul 2025 21:44:41 +0000

Robin Sloan asks: Is the doc bot docs, or not?

He relates a recent experience he had with an AI chatbot where the chatbot provided a response for his question that was incorrect despite being provided from the authoritative context of the official Shopify documentation.

Reacting, Sloan considers:

I suppose there are domains in which just taking a guess is okay; is the official documentation one of them?

I vote no, and I think a freestyling doc bot undermines the effort and care of the folks […] who work […] to produce documentation that is thorough and accurate.

As a writer of official documentation, I agree.

Most documentation sites are part of the official service agreement for a product, often serving as the warranty for how the product is supposed to work, and which uses are supported by the company. Meanwhile, most LLM technology is prone to hallucinations (not to mention prompt injection), even if constrained by the system prompt.

If you provide an AI-powered chatbot on your documentation site, you must consider how to make it a productive and accurate experience for your customers…

Imagine providing a more powerful search for your documentation, but instead of a search returning no results, it summarizes a best guess. Sometimes that can work, but sometimes that means you provide irrelevant information. The nature of generated text is that it seems far more authoritative than a clearly irrelevant link in a list.

It’s easy to add a bot to your docs site as an experiment, but if you do so, it’s important to be intentional about it, and follow up after deployment. If you want to add an AI-powered chatbot to your documentation site, before and after you add it, consider the following:

Add disclaimers

If the content output by your AI-powered chatbot might be inaccurate, consider indicating that your bot is in beta, or otherwise not officially supported.

If the service or license agreement that customers sign for your product includes your documentation as an official guarantee of product functionality, it might be a good idea to check with your legal team (if you have one) to ensure that the responses from the AI-powered chatbot that you deploy are not subject to the same guarantee.

In addition, you might want to add an information disclosure as well, letting people know how and whether the information they provide to the chatbot is collected and used.

Strategically supplement responses

Presumably, if you’re providing an AI-powered chatbot on your documentation site, it’s configured to supplement and source responses from the official documentation using retrieval-augmented generation (RAG).

When you identify the sources to use, you might want to consider whether those sources should include more than just the official product documentation. If you have a community forum site full of customer context (but less up-to-date or accurate information), or a technical blog full of use cases (and similarly potentially outdated), you might want to make those available to the chatbot.

Ideally you’d have some way of adjusting different weights (though it might be limited to temperature or other vague setting names). If you have multiple sources that you want to draw from, being able to indicate which ones are more authoritative than others, or at least prioritizing more recently updated content with the expectation that it’s going to be more accurate.

Test outputs

Set up automated and manual testing of the outputs for accuracy and consistency. The best way to do this is to build a “ground truth” dataset of questions and answers. After you have a canonical dataset of common questions and accurate answers, you can evaluate the output of the AI chatbot against that dataset.

Perform evaluations generally (is the output correct?) and over time (is the output still correct?) to ensure that the quality of responses doesn’t drift.

If you’re new to evaluating the output of LLMs, many chatbot providers like Kapa provide resources that outline these tactics, such as in Kapa’s doc Conversation review best practices. Intercom seems to provide a set of automated performance metrics for their FinAI chatbot which, if you trust them, you can use to review the quality of responses.

If you’re not using one of these providers (or even if you are), the field of data labeling, especially for NLP data, has a wealth of resources available. Microsoft provides a detailed list of metrics for evaluating LLM-generated content.

You can also automate testing of outputs with a tool like Evidently, incorporating testing into your CI/CD pipeline.

Monitor input and output

Beyond testing the quality and accuracy of the chatbot responses, you also want to monitor the questions being asked and the responses offered. The data from AI chatbot interactions can be used almost like a combination of search term data and community forum content — identify missing content, misaligned mental models, inconsistent terminology, customer use cases, and more that you can use to improve your content.

The monitoring process isn’t just about improving the accuracy of your content, but also its relevancy to customers.

Review customer inputs to understand use cases that customers have for your product, including the terms that they use to describe functionality.
Review the outputs from the chatbot to identify situations where the chatbot hallucinates information or mixes up concepts in its answer.

You can then take action in your regular documentation process to address anything that you find:

Enhance existing documentation with new end-to-end examples that draw from customer use cases.
Update content to include different terms that customers might use to describe particular functionality.
Create content to address missing functionality, such as by adding missing documentation about supported functionality, by posting an explanation about the unsupported functionality on the community forum, or something else.
Perform a terminology audit and update informed by unclear responses from the chatbot, identifying and resolving cases where you might be using the same term to refer to different concepts or functionality in your product.

Chatbots aren’t docs, but you can use the interactions with them to make your docs better.

My writing setup

Sun, 25 May 2025 22:35:28 +0000

I do almost all of my writing in Google Docs and VS Code. Occasionally I’ll take longhand notes or jot things down in Apple Notes, but the real content that I write at work and for this blog is in one of those locations.

My VS Code setup

When I’m working in VS Code, I use several extensions to help make my work as easy as possible.

Code Spell Checker

Misspelling things in strings in code and when writing in an IDE without built-in spell check can be a challenge. This extension adds a straightforward and easy-to-use and configure spell checker to VS Code. It also advises on some grammar rules, non-intrusively.

Code Spell Checker from Street Side Software

Markdown All in One

This extension is best-in-class for writing Markdown. Get autocomplete for links, auto-created table of contents, and automated renumbering of lists.

There’s probably more in there, but those are the game changing features for me of this extension.

Markdown All in One from Yu Zhang

Markdownlint

Like any good linter, this extension helps enforce style standards when writing Markdown, ensuring that your syntax is consistent and correct throughout your documents. Also critical? It highlights when link fragments (like anchor links) are invalid.

I don’t like to see linting rules while I’m writing on a specific line, so I use the following settings to better manage that:

"markdownlint.run": "onSave",  
"markdownlint.focusMode": 5,

markdownlint from David Anson

TODO Highlight

If you add TODOs to what you write, you need to be able to find them again so that you can fix them.

TODO Highlight by Wayou Liu

Indent-Rainbow

After writing in ReStructured Text without a linter for over a year, and with any code that I write being in Python, being able to see exactly what indent level I’m at is crucial for valid markup and syntax.

indent-rainbow from oderwat

WordCounter

I forgot I had this extension installed because it’s unobtrusive. I use it to get a quick glance of how many words I’ve written for a blog post (and provide an early indicator that I’m writing an epic by accident).

WordCounter by Etienne Faisant

What I don’t use

I don’t use any git extensions, and I don’t use Vale.

I have GitLens installed but I’ve never actually used it because I prefer to use GitHub Desktop as my primary interface with changes that I make with git.

I don’t use Vale because I don’t use a style guide when writing my blog posts. When I’ve worked in docs-as-code environments, the style guides hadn’t been codified into Vale rules and programmatic style checking was not a priority.

Other VS Code settings

To make sure my files always get saved:

"files.autoSave": "onFocusChange"

To manage switching between git branches even easier:

"scm.workingSets.enabled": true

To make the text that I write easier to read:

"editor.wordWrap": "on"

To see every single space in my writing and eliminate accidental double spaces:

"editor.renderWhitespace": "all"

This was especially critical when writing in ReStructured Text, because an extra space could break a list or disrupt the formatting of an entire doc. Markdown is more forgiving, but I’m in the habit of enabling this setting ever since.

Google Docs settings

These are a few settings in Google Docs that I use all the time. Most of these can be set by going to Tools > Preferences:

Disable show spelling and grammar suggestions. I find it distracting to have squiggly lines appear when I’m typing a super rough draft or taking notes.
Disable automatically correcting spelling and capitalizing words. I find it especially distracting to have my words change as I’m typing them. I want typing in a Google Doc to feel almost as seamless as writing out longhand, and removing autocorrect from that process is vital.
Disable smart quotes. They’ll screw up any formatting that you copy and paste into a code sample, and I don’t need fancy typography.

I also use keyboard shortcuts constantly, whether it’s reformatting text to be a header or to match a standard style, or to paste without formatting.

I’ve been delighted with the Copy as Markdown and Paste as Markdown options. Despite the exact markup not being what I always choose to use, it makes formatting documents for tech review or codifying hasty drafts for publishing a lot faster.

Other shortcuts for writing

Generally, on my computer I tend to have a few keyboard shortcuts set up for the purposes of text autocomplete:

A text replacement rule to spell my first name correctly. Sometimes I type too fast and misspell it, and that would be embarrassing.
A text replacement rule to spell my company name correctly. Almost no matter what the company name is, it’s nice to have the failsafe.
A text replacement rule for my email address. It’s faster to type @@ than it is to type my full email address.

I’ll also set up other rules to catch common misspellings that I make — endpoing, documetnation — as well as finish long words or phrases — authz for authorization, authn for authentication, descr for description, and more.

I’m not big on personal optimizing, but I do find it helpful to make the process of typing as effortless of a translation from mind to document as writing longhand notes for myself.

A career bucket list for technical writers

Fri, 19 Apr 2024 08:53:07 +0000

What’s next for you in your career? It’s tempting to focus on the bare minimum — staying employed — but identifying new areas of professional development or focus can help you grow your career and find whatever enjoyment you can from the capitalist toil that is an obligation of modern life.

While I was at Splunk, Susan St. Ledger gave a talk about approaching her career with a bucket list. I’d defined my career values, but a bucket list for my career was the perfect way to complement my career values while still growing my expertise.

Even if you have the same title, you can have different experiences with your career and learn different skillsets if you seek out different team sizes, company stages, reporting structures, and working environments.

Identifying that variation has been crucial for me as I’ve stared down a lifelong career doing “just writing”. Did I really want that? Thankfully, technical writing doesn’t look the same everywhere, and that variation is what keeps it exciting for me.

In chatting about next steps in a technical writing career with a mentee, we came up with the following list of experiences and job situations that could be on a bucket list for technical writers.

What types of experiences and job situations might make sense on a bucket list look like for technical writers? I’m still developing my own, but I wrote up this list to serve as inspiration:

Tooling:

Migrate a doc set to a new platform or tool.
Implement a doc feedback collection tool.
Implement an in-product tutorial tool, such as WalkMe, Pendo, Gainsight, or similar.
Maintain and customize a documentation platform.

Writing process:

Write in a docs-as-code workflow.
Write structured content like with Doxygen or DITA-based tools.
Write in a different markup language (HTML, MediaWiki, Markdown, Restructured Text, AsciiDoc, DITA XML, etc.).

Writing tasks:

Write a documentation set from scratch.
Write API documentation.
Write technical content marketing and/or blog posts.
Write in-product tutorials and tours.
Write tutorials.
Write stellar reference content.
Reorganize or rearchitect a documentation set.
Write UI text and error messages.

Writing and design tasks:

Create detailed diagrams.
Design graphics and icons.

Collaboration or stretch opportunities:

Collaborate with marketing on content.
Do product development, such as by writing an interactive in-product tutorial, or committing code to the product.
Design a documentation site.
Perform user research.
Do customer support.
Work with developer relations/advocacy on content.
Develop best practices documentation with the field (sales engineers, professional services consultants, etc.).
Partner with the web team on SEO improvements.
Start a naming council to codify and manage product and feature names.

Writing standards:

Write a documentation style guide.
Write a UI text/content design style guide.
Define audiences or personas for the docs.
Work with editors.
Define templates for reference and other content.

Writing strategy:

Define documentation strategy and vision for a product/company.
Make a business case for documentation.
Develop documentation analytics KPIs and success metrics.
Define and implement a triage process.
Formalize and mature documentation planning processes.

Personal brand:

Present a talk at a conference like STC Summit, IAC, Write the Docs, LavaCon, Button or others.
Speak on a podcast.
Participate in a panel discussion.
Write about documentation for the company blog.

Writing-relevant roles:

Be a manager of a large team.
Be a manager who grows and scales a small team.
Be a team lead.
Be a lone writer.
Be a minion at a large company.
Mentor a teammate.
Become an editor.
Be a scrum master.
Be a content designer.

If you’re looking for what’s next, whether at your current company or somewhere new, I hope this list helps you consider what you might want to accomplish next!

Docs as code is a broken promise

Wed, 10 Apr 2024 21:27:12 +0000

Docs as code is a much-vaunted workflow and toolchain for writing, publishing, and maintaining technical documentation — but in practice, docs as code doesn’t deliver on its promise.

What is docs as code?

According to the Docs as Code page in the Documentation guide for Write the Docs:

Documentation as Code (Docs as Code) refers to a philosophy that you should be writing documentation with the same tools as code:

Issue Trackers

Version Control (Git)

Plain Text Markup (Markdown, reStructuredText, Asciidoc)

Code Reviews

Automated Tests

This means following the same workflows as development teams, and being integrated in the product team. It enables a culture where writers and developers both feel ownership of documentation, and work together to make it as good as possible.

This is promising — shared ownership of documentation and a shared goal to make documentation as good as possible by sharing the same tools and processes.

All too often, however, I find that this philosophy is adopted as a set of tools, and the processes and integration are ignored. Even if you have processes in place, the tools you use to do docs as code make it easy to circumvent the processes, even unintentionally.

The promise of docs as code

If your documentation workflows follow the development workflows, as a technical writer, you can get your job done more easily.

If you do docs as code…

Documentation reviews can take the same format as code reviews—a pull request—making it easier to get reviews from engineers.
Code samples can be automatically tested, or better yet, pulled in at build time from the product code.
Doc changes can happen in sync with the product changes if docs are in the same repo, streamlining the release cycle.
Documentation can be versioned, and you can keep track of when and how a given page was last updated.

The pitfalls of docs as code

Writing content isn’t the same as developing code, and as a result, doing docs as code has pitfalls:

Let me explain…

Git is confusing

To do docs as code, writers need to learn how to use and troubleshoot Git. And Git isn’t simple — Julia Evans made an entire Zine about it and has been working on a longer series about Git concepts to make it much clearer.

There is a deep level of complexity with Git, and while you can go a long way only knowing how to git fetch, git pull, and git merge, it’s easy to get into an unexpected state with your cloned Git repo, local branch, or disastrous merge conflict resolution decisions. In those cases, often the best troubleshooting is starting over — and that’s not a great experience!

Processes must be defined and reflected in Git workflows

When you use a docs as code workflow, you need to codify your docs processes and instantiate them in your Git workflow. So you not only need to define the following:

When to publish doc updates
How to release doc updates
How to coordinate a docs release with a product release

You also need to define Git best practices for your team about how to manage those, such as:

Whether to use release branches, or merge pull requests frequently but publishing infrequently.
Whether to use Git rebase or Git merge to maintain Git history in a given branch.
Whether and how to use feature branches and pull requests.
Whether to squash merge pull requests to main.

Even if you manage to define best practices that your team is committed to following, there isn’t a way to force your documentation contributors to adhere to all of these best practices. Due to the lack of enforcement of these best practices, you can easily end up in a situation where writers follow slightly different practices based on what their tools make easy to do.

As just one example, I spent several weeks merging all my pull requests to main, instead of squash merging as was the best practice. At some point, I’d needed to do a regular merge and preserve all my branch commits on main, and GitHub “helpfully” remembered my last-used setting.

Because the best practice was just a practice, and not enforced, it took me several weeks of cluttering up main with my chaotic branch commit history before I realized what was happening. In the meantime, the best practice of squash merging, meant to maintain a clear association between a PR of feature changes and a Jira ticket, was completely ignored with no warning.

Tools to write docs can be inconsistent

In a typical docs as code environment, you write locally and push your changes up to a central repository. Unlike a typical content management system (CMS) like WordPress or Drupal, or software like MadCap Flare, the writing environment isn’t shared — only the content is the same.

Instead, the writing environment is relatively uncontrolled by default — a writer can choose to write their content in whatever system they want, so long as they can commit text to the Git repository.

A writer contributing to the documentation can write in a basic text editor like Sublime Text, an extension-filled Visual Studio Code setup, in vim, or some other tool — even the GitHub or GitLab UI!

Because only the end result matters to docs as code, different writers and documentation contributors can use different tools with different settings and functionality. However, these environment inconsistencies can lead to inconsistent content quality and style, and make issues with content development more difficult to troubleshoot.

If you use a text editor that doesn’t have a spell checker installed, you can easily introduce typos into the documentation — but only the topics that you contribute to. If your colleague doesn’t use a system supported by the automated style checker recommended by your team, the style of their documentation topics will deviate from company style.

Merge gates and build checks are great… if you have them

One advantage of docs as code is the extensibility and automated nature of the workflows. Because you can treat documentation content like code, you can implement some code-like practices, like checking the validity of your documentation markup when you propose changes to the documentation with your pull request (a merge gate) or when you build the documentation site (a build check).

However, to take advantage of these capabilities, you often need to build the tools and checks yourself, or get your documentation platform team to build them for you — right after they work on the rest of the backlog of improvements for the customer-facing documentation site.

Some companies put a lot of effort into engineering efficiency, with entire developer experience teams devoted to improving engineering workflows. Meanwhile, documentation teams are often lucky to get one engineer to work on the documentation site itself, let alone help enable any special docs as code workflows. This makes sense for the business, but it makes it difficult to take advantage of what docs as code can do for you.

For example, you could set up merge gates or build checks to perform documentation quality improvement tasks like the following:

Optimize image size
Lint content to standardize and correct markup
Check for broken links
Test code examples
Validate adherence to the style guide
Check spelling and grammar

Some of these might make more sense to implement in your writing tool directly (such as with Visual Studio Code extensions), but then you would need to require writers to all use the same writing tools configured in the same way.

Without resources to implement custom merge gates or build checks for documentation quality, documentation writers have the added process burden of opening a pull request, requesting reviews and approvals, and miss out on the potential advantage of automating tedious tasks. And if their content is in the same repository as the product code, they also need to wait for a bunch of unrelated tests to pass before they can merge any doc updates.

Reviewing content in the repository is hard to read

A much-touted benefit to docs as code is that the content is in the same tools used by engineers, which makes it easy to get technical reviews done in GitHub, GitLab, or your Git provider of choice.

Unfortunately, documentation reviews in a pull request can be confusing. Just like reviewing UI code is difficult if you only have access to the source and not a staging environment, if you can’t provide your reviewers with a staged or preview version of the content, you might get a technical review full of confused comments instead of helpful feedback.

An engineer might see that you deleted lines from a file, not realizing that you deleted them because they were in the wrong place, not because the information was wrong.

Someone else might comment that you need to provide a note on several pages, not realizing that you did that — using a single sourced content reference that isn’t obvious to someone who doesn’t write documentation.

Parsing a documentation pull request and providing helpful comments requires the reviewer experience the content the same way that a future customer will — but a pull request only provides the raw, marked up text.

Upholding the docs as code promise

In its current state, with these pitfalls I’ve outlined (and more), I find it problematic to recommend docs as code as a one-size-fits-all solution for all your technical writing CMS needs. Docs as code is a workflow with processes and tools, and therefore requires investment, maintenance, and a decent amount of custom tooling to get true value out of it.

It’s possible that using a well-resourced static site generator like Docusaurus or MkDocs as part of your docs as code workflow might address some of these pitfalls, but from my impressions writing, building, and deploying this blog using the Hugo static site generator, it’s unlikely. I’d guess that those give you extensions and flexibility to make a static site work well for documentation, but they don’t solve the core writing experience challenges of a docs as code workflow.

If you’re considering implementing docs as code workflows, you can choose the degree to which you want to adopt the practices. If you require a branch, pull request, and all build checks to pass to fix a typo, the time it takes to fix a typo could easily triple. On the other hand, with that level of overhead, you gain ultimate auditability of documentation changes — the what, when, why, and by whom.

When considering docs as code workflows today, I often see what my friend referred to as the “most extreme option” — a writing team and toolchain that has adopted all the practices from code development. I think to uphold the promise of docs as code, we could do to add a bit more of the docs back into the workflow.

As Fabrizio Ferri Benedetti put it in his post, The pros and cons of using Markdown ¹:

you want the docs to be the product […] But be very careful about the product not being, for example, the pipelines or the site you’re gonna render.

There are newer CMSes on the market, like ReadMe, Heretto, Paligo, and others. From my extremely limited experience using one of these CMSes, it seems like the writing experience is much simpler, and there is a way to write Markdown without needing to use developer level tools.

Unfortunately, it seems like these tools also still need to mature and borrow more of the practices that do work in docs as code, like being able to compartmentalize documentation drafts and release specific changes at a specific time.

It could be as well that we are early enough in the docs as code ecosystem that the tooling hasn’t yet matured—but Anne Gentle’s book Docs Like Code came out in 2017, so this concept has been around for some time.

I sincerely hope that the content management systems improve, or that we see more standardized toolchains with guardrails that enforce best practices for doing docs as code.

The easier it is to implement docs as code — consistently, without much room for individual writer error — the easier it is to focus on producing high quality documentation, together. This is exactly why large companies have entire engineering efficiency teams.

What would it take to get a writer efficiency community to focus on a spectacular docs as code experience?

That post is a follow-up of his comments in the webinar, Pros and Cons of Using Markdown for Technical Documentation Panel Discussion, which covers some of the same ground as I do in this post (and beyond), such as things about availability and scalability of the content for developers contributing to the content, and an illusion that implementing such a workflow can be free. ↩︎

What about GIFs instead of screenshots?

Sat, 06 Jan 2024 18:16:57 +0000

After I published Should you add screenshots to documentation?, I got some comments from folks who prefer GIFs to screenshots because GIFs can more clearly show how to use a complicated user interface.

I agree that GIFs are cool and useful, but they’re also MUCH harder to keep up-to-date than screenshots and have extra accessibility considerations if you decide to use them.

WCAG level A standard requires that:

For any moving, blinking or scrolling information that (1) starts automatically, (2) lasts more than five seconds, and (3) is presented in parallel with other content, there is a mechanism for the user to pause, stop, or hide it

If you use a GIF, you need to consider those accessibility constraints, as well as the standard consideration when providing visual task steps: Can someone complete the task successfully if the screenshot, GIF, or video is unavailable?

So if you decide to use GIFs instead of screenshots, keep in mind these extra considerations:

Can someone pause, stop, or hide the GIF?
Does the GIF stop moving in less than 5 seconds?
Did I write effective alt text for the GIF?
Is the source file for the GIF stored somewhere accessible to the rest of my team?
Do we have clear and consistent style guidelines for creating GIFs?

Displaying content as a graph: An exploration

Thu, 28 Dec 2023 20:20:27 +0000

Most web content is designed to display with a strict hierarchy, tree-based or otherwise. What if it wasn’t?

What does it mean to display content as a graph?

Instead of being displayed as a strict hierarchy, content could be placed in multiple categories within a hierarchy, using a polyhierarchy, or displayed as a series of interlinked nodes in a network—a graph.

Displaying content as a graph could visually communicate the relationships between ideas, tasks, and topics in technical content. A hierarchy also does this, but doesn’t always match the actual relationships between ideas and concepts in technical content.

A graph could offer a more flexible presentation, but content is usually displayed hierarchically on the web. Why is that?

Why hierarchies are common

Hierarchies are common because they match the typical way of explaining a new concept or topic — you start out with an overview, briefly introducing several important things and how they fit together, and then provide additional information about each thing later on.

For example, if I want to teach you how music is made, I’m not going to start with an intense explanation of how an instrument works to create sound. Instead, I’ll start with a high level discussion about the components involved in music: a song, an instrument, scales, key signatures, tempo, and then move into more complex topics like how to write music using those components.

How to make music
- What is a song
  - What is a hook
  - What is a verse
  - What is a chorus
  - What is a bridge
- What is an instrument
  - Different types of instruments
    - String instruments
    - Woodwind instruments
    - Brass instruments
    - Percussion instruments
  - How to tune an instrument
- What are the components of music
  - Types of scales
  - Melody and harmony
  - Tempo and time signatures
  - Key signatures
- How to write music
  - How to write a hook
  - How to write a verse
  - How to write a chorus
  - How to write a bridge
- How an instrument works
  - What is sound?

It’s fairly easy to scan a list of items and follow the hierarchy that represents a logical progression of information and knowledge.

Hierarchies also mimic a traditional way of displaying information—as a book, with chapters that represent hierarchies of content.

Why use something different?

Hierarchies are common and familiar. So why use something different?

Serve multiple mental models

Presenting content in a hierarchy provides an implicit mental model to the reader. By ordering content, there is an implication that the information is listed according to its relative importance and in the order which you should learn the information.

In the case of the music example, because of the hierarchy presentation, that means that you should learn the component parts of a song before learning about instruments, and before learning about scales, key signatures, or tempo.

If you display content as a graph, you can remove some ordering implied by a hierarchy:

By expanding a hierarchy, either by placing content in more than one location, or by removing the ordering involved in a hierarchy, you can serve multiple mental models used by your readers and thereby improve findability.

An article from Page Laubheimer for Nielsen Norman Group¹, points out some benefits of categorizing content in more than one location:

A polyhierarchical IA is a structure where an item exists in more than one place — that is, it can be reached following several category paths. … Ideally, you want to create categories that accurately describe your content and products, are extensible when you add new topics or products, and match your users’ mental models (which often vary from one person to another).

And that is a crucial point — we often write documentation with an audience in mind, but there is always variation in familiarity and background.

For example, because I learned data analysis with a language that performs aggregation with the statistical transformation command ², I might look for the SQL function GROUP BY alongside the COUNT() and SUM() function reference.

If content could be displayed as a graph, depicting different relationships across categories and commands, the content can reflect many mental models. The content could become the diagram.

Write better documentation

A persistent challenge for technical writers is to focus on writing user-centric documentation rather than feature-focused documentation.

When you consider content with the complexity of a user journey, with all the requisite branches and forks, the limitations of linear hierarchies begin to show.

It could be easier to write documentation if you didn’t have to consider where the content belonged in the navigation or the table of contents, but instead considered how it related to other pieces of content that already existed. We do that already, but must always create a hierarchy that considers order.

But sometimes order doesn’t matter. Sometimes order is different depending on the audience. Hierarchies are inflexible.

Without being constrained by a hierarchy, we could place documentation in a structure more mindfully, answering questions like:

What relationships do you want to draw for the reader?
When and where might the reader encounter the tasks described in this topic?

David Ryan at Red Hat has tried this, pointing out in a tweet that:

A few of us used a graphing tool at Red Hat to show content relationships, as we experimented with moving away from “linear book” and tried to match content with the user journeys.

For the music example, we can acknowledge the fact that some readers might be familiar with writing lyrics, but not familiar with the concept of key signatures or scales. Other readers might be familiar with songs, but can’t keep tempo easily, so want to explore that content in more depth than others.

Improve content reuse

If you don’t have to draw strict hierarchies, it could be easier to reuse content. If your content is pocket-sized and specific to a concept or task, recontextualizing it would be as simple as adding another link to the graph.

In a hierarchy, to reuse content you often have to duplicate the content to another location—through single sourcing or otherwise—to make it logically relevant.

The flexibility of a graph means that you can use labels or links to draw connections across content, rather than duplicating it to another location in a hierarchy.

Improve machine legibility

Content that is not just stored as a graph but also written for a graph would be even more legible to machines—the website crawlers indexing content for use in search result ranking algorithms and capturing datasets to train machine learning models.

Most of that content is already crawled as a graph, with the robots following link after link, indexing the page contents as well as the links between them, building a graphical representation of the web.

Writing with those relationships in mind could help amplify the connections across content, making those connections more legible to the machines while exposing them to the human readers as well.

Depending on how you invest in the quality of your knowledge graph, the search benefits could create a competitive moat for your documentation site—provided the content exists to support the search terms.

Why don’t we display content as a graph?

There are a number of benefits to displaying content as a graph, but very few websites do so. Why is that?

It’s difficult to design

Websites are based on pages — and it’s difficult to expose the underlying structure to readers in a coherent way.

Having a navigation menu at the top or the side of a page is an extremely familiar pattern, as well as “breadcrumbs” to help people maintain an idea of where they were before.

There are a few examples of content graphs on the web already, and they follow some different patterns:

Andy Matuschak has created a form of a graph with his Evergreen Notes ³, which open links next to the content that you’re interacting with, and minimize along the way. In the case of Evergreen Notes:

there’s no index or navigational aids: you’ll need to follow a link to some starting point.

It’s a similar design to what Obsidian implemented ⁴. Obsidian also uses a graph and node-based architecture to relate local text files on your computer with one another. It takes the graph visualization to the next level, but I feel that graphs of this scale aren’t super discoverable, despite being explorable, because they’re so overwhelming. Obsidian also recently introduced tab stacks, another innovative way of visualizing content all at once.

Another site is the C2 wiki, which was described to me as “a discussion site that happened to be shaped like a reference guide.” It essentially creates a map for you as you navigate through it, with the links you open floating next to the origin.

I love this design as a way of keeping track of a specific journey through the content, enabling the reader to build their own workflow out of the content, but it raises a challenge when it comes to sharing links.

In the screenshot above, my pathway ended on the DepthFirst page, but if you open that link, you see only the page—none of the journey that led me there, and none of the nodes in the graph that you might want to explore alongside it are visible—only a few links in the page itself.

These two sites expose the main design challenges:

What do you display on the landing page, or entry point, into a site like this? Most readers encounter documentation from search, but many still browse to the site itself—especially prospective customers or employees looking at the company website.
How do you make the graph visible when viewing a discrete page (or node) in the graph? How do you make the context of the current page visible to the reader?

The landing page option might be to do something similar to many hierarchical documentation sites—display a search bar and some key “entry point” nodes with select branch or dependent nodes displayed.

Making the graph visible could be an interesting design challenge, depending on what you choose to do:

Ignore the graph and show only the node, leaving the links as the only way to navigate to other nodes (and follow existing web browsing patterns without adding new ones).
Preserve a specific journey when sharing a link to a specific page, and offer a “historical” graph view, similar to what Andy’s Notes does.
Provide a zoomed-out view option, where you could minimize a page and view the nearby graph nodes:
Something else entirely!

It’s confusing for readers

Hierarchical content is a dominant web format, and as a result, it’s familiar. Familiar design and layout patterns are easier to navigate because you don’t have to learn anything about how to use the site, you can just navigate using the patterns you’ve already learned — scan the navigation bar, scan the table of contents, and skim the headers on the page or use ctrl+f to search for a keyword in the text.

As Mark Baker points out in Every Page is Page One, people are information foragers. The readability of a graph and the sheer amount of content that might exist in one might not lend itself well to the scanning-first habits that we’ve evolved as web readers.

Instead, if you display content as a graph, readers are confronted with an unfamiliar pattern that could require them to learn new buttons and interactions in order to find what they need. That’s a hindrance to readers of technical documentation, because their goal is to answer a specific question, complete a specific task, or learn about something — not learn how your website works.

In addition, displaying content as a graph lacks a clear hierarchy, which causes another big problem: there’s no clear starting point.

As Kelley Gordon points out in her article for Nielsen Norman Group, Visual Hierarchy in UX: Definition:

If you struggle to find focus on a screen, it’s likely that the layout is missing a clear visual hierarchy.

The page’s visual hierarchy controls the delivery of information from the system to the end user — it lets users know where to focus their attention.

Arriving at a webpage without visual hierarchy and only a graph with nodes can be disorienting for readers.

Most readers would then use another form of exploration, such as search, to find what they’re looking for.

Because a graph lacks a clear visual hierarchy and can be overwhelming, the designs of Evergreen Notes or the C2 Wiki make sense because they reveal the graph as a journey as you navigate the page. Other sites that use graphs don’t usually make them visible as you navigate.

One exception is the classic site HyperPhysics Concepts⁵, which uses a visual graph representation with clickable links in order to help you navigate the site:

This graph is a hierarchical and ordered graph, but less hierarchical than a traditional table of contents, and made visible to the reader. Each node of that graph is clickable, opening a new page with content, or sometimes a new graph:

The Hyperphysics site makes the graph visible but not at the expense of the content. The graph can be annotated and serve as another form to communicate the relationships between the concepts described in the content.

Another exception is the site Every Noise at Once. The data sources for it are sadly now defunct, but the site itself displays not a graph, but a collection of genres laid out as though there were node connections visible.

It’s the idea of a graph, but with no true connections. The site itself refers to it as a scatter plot — so there are vague associations, but no true links. If you go deeper into the graph and select a genre, you see a similar scatter plot of the artists in that genre:

Sites with hierarchies also use breadcrumbs as a way to show you where you are in the overall site map. Content that you interact with purely in graph form would instead need to either construct dynamic breadcrumbs based on the path through the content that you took, or take a different approach entirely, like relying on category tags.

Every Noise at Once, with the flexible graph-like exploration format, maintains a breadcrumb that effectively shows the nodes surrounding the one that you’re currently visiting:

Ultimately, there are a variety of options for displaying content visibly in a graph, but most of them either retain a hierarchy to assist readers in scanning and foraging for information, or have no visual hierarchy and force the reader to explore, use search, or give up in the face of overwhelming options.

It’s too flexible

Content graphs aren’t a new concept. Displaying content as a graph might be rarely done, but storing content as a graph is a common practice—usually referred to as a knowledge graph.

As Sarah O’Keefe discusses for Scriptorium, there is a cost to knowledge graphs. One of those is the challenge to writers:

We have to think about the various content or data objects, understand how they relate to each other at the knowledge graph level, and then bring them together into a coherent experience, whether a webpage, a document, or something else entirely.

That’s a lot to consider. Writers using DITA (Darwin Information Typing Architecture) and XML-based structured authoring solutions might already be familiar with these challenges, and the chunking approach to reduce content to its relevant bits which then need to be reconstituted into a coherent experience on a webpage.

I haven’t worked in a DITA environment before, and most of my experience is with tools that don’t use a database to store content, let alone a graph database. Many docs-as-code tools require a hierarchy of sorts to dynamically generate a table of contents (Sphinx is one example) or to provide some sense of order to the flat file structure of a repository.

When it comes to organizing content, it could be freeing to consider all the different ways a topic could be relevant to a reader, but it could also end up exhausting, deciding where to draw the line.

For the music example from earlier, a page, or graph node, about “How to write songs” could be linked to:

How to write a bridge
How to write a chorus
How to write a verse
How to write a melody
How to write a topline
How to write a hook
how to write lyrics
how to write a harmony
how to use a digital audio workstation (DAW)
and more…

However, the flexibility of a graph could add chaos. If this content could be linked to other content, what should it be linked to? What other nodes are relevant? A graph could quickly turn into a sea of possibilities, rather than intentional choices about relevance.

At a certain point, the attempt to avoid missing a relevant link, or to try to make your content as discoverable as possible through a knowledge graph, would dilute the explicit mental models and diagramming potential possible with a graphical presentation of content.

Instead of replicating some mental models, the graph could instead start to represent all possible permutations of a mental model, and stop representing anything of value at all.

But the act of considering all the different locations where your content might be relevant, what other entry points might exist, could result in more thoughtful documentation content. You can consider it as part of an entire system. By considering the content as part of a graph, you can more easily treat the content as part of an expansive surface, rather than a discrete element on a specifically ordered list.

By contrast, because a hierarchy is strict, it forces choice. If this content had to exist in one place, where is the most relevant place?

That one choice can be daunting, especially when choosing what the top of the hierarchy should be.

In a discussion in the Write the Docs Slack community, Wouter Veeken points out that one difficulty with hierarchies is deciding what to put at the top of it, asking if anyone has a “general method for deciding which attribute should be the top level of the tree?”

Therein lies the tradeoff. For some content, choosing a location in a hierarchy is an arbitrary choice. For other types of content, it reifies a relevant order to a task.

For example, if you want to write a song, you can start with a melody, a harmony, or lyrics. To write lyrics you can start with a chorus, or a verse, or a bridge. None of those have an order. But if you want to produce a song, a song must be written first. In that case, order matters.

This is a key advantage of having a graph structure—everything is a set of relationships, there is no designated “top” of the tree to identify. You could even choose to display highly-related topics differently than others, and use the information provided by the graph structure itself to determine what content is most valuable.

It’s difficult to write chunked content

Those are the challenges with structuring the content, but there are also challenges with writing the content.

Carrie Hane points out in a discussion about Knowledge Graphs on Sarah O’Keefe’s LinkedIn that:

adoption of structured, decoupled content is still needed to adopt knowledge graphs. And that is a huge leap.

I agree with that completely. Each piece of content in the C2 wiki or the Hyperphysics site was a discrete component of content — a definition of a concept, for example. That type of chunked content fits well within a knowledge graph.

Technical content written for a hierarchy, or with an implied order, can often end up quite long and detailed — perhaps with one page describing everything you need to know about something. To continue the music example, you might end up with a page like:

How to write a song
- About song components
- Write lyrics
  - Write a verse
  - Write a chorus
  - Write a bridge
- Write a melody
- Write a harmony
- Write a bassline
- Write a percussion track

That page is full of task-based content that could be easily separated into different graph nodes, but it might also include definition content mixed in, like:

What is a hook
What is a topline
What is a key signature
What is tempo

That definitional content would also get broken out into specific chunks. All of a sudden, instead of writing an end-to-end topic all about songwriting, you’re writing tiny chunks of content that might not be as interesting, compelling, or cohesive to write or read.

Sparse content and dense content don’t scale in a graph

Graphs are only useful if there are lots of connections. A graph with only a few nodes doesn’t communicate much as a diagram of a mental model:

On the other hand, too many nodes is impractical and also doesn’t communicate much as a diagram of a mental model:

Visible graphs: The future of displaying content?

Ultimately, it’s unlikely that visible graphs would become a standard practice for displaying technical content. It’s a complex design and rendering problem, and while there’s a chance that the novelty might get someone to click around in a graph for a bit, someone with a specific question in mind wants to be able to scan for information.

A hindrance to scanning is volume of content. Given the amount of content on many documentation sites, and that people are accustomed to navigating in digital devices, most people default to using search as their first choice for navigation.

If everyone is using search to navigate, displaying content as a graph wouldn’t make much sense.

Instead, implementing a knowledge graph as the background structure for a documentation site, with clear guidance about how to connect the nodes of the graph, has the potential to improve search results for readers and allow flexible categorization for writers.

A knowledge graph lets you have the best of both worlds. As Manav Modi from ProductX points out in How is Airbnb optimising Search and Discovery using Knowledge Graphs? 🎯:

The graph can also be hierarchical, with high-level concepts branching down to more specific details, allowing for a streamlined data organization.

An alternative that can preserve the graph conceptually but not visually is the tile-based approach common on many documentation landing pages, with sort and filter options:

One compelling example is the Redis command docs, as shared by @matheusfelipeog in their beautiful docs repository.
Another one is the GitHub Docs which combines a tile-based presentation with a table-of-contents hierarchy, as well as recommended pages based on “getting started” or “popular topics”.
The Digital Ocean docs also use a tile-based approach combined with a table of contents.
Mode uses tags on their resources page, letting you filter by type of content and/or the subject of the resources.

In these models, the graph isn’t made apparent, but the node relationships still exist through tags or content types and can be explored without using a table of contents. This approach lets you gain some benefits of visually displaying content as a graph without the most troubling challenges—an unfamiliar and challenging browsing experience for readers.

No matter how you visually present your content, the act of placing content within an information architecture and structuring it is valuable and reifies a mental model for the readers and the writers.

It’s impractical to make a graph visible to readers, but I still think it would be cool.

Polyhierarchies Improve Findability for Ambiguous IA Categories ↩︎
SPL, or Search Processing Language used by Splunk software. ↩︎
Thanks to Kimberly for sharing this site with me. ↩︎
Thanks David Ryan for sharing this site with me. ↩︎
Thanks Scott Centoni for sharing this site with me. ↩︎

Should you add screenshots to documentation?

Sun, 10 Dec 2023 21:15:21 +0000

Screenshots in documentation can be a contentious topic — some people really like them and think they add a lot of value, while others dislike them due to the maintenance burden and accessibility issues.

As a technical writer, I avoid adding screenshots to documentation as often as possible. In my mind, an outdated screenshot is one of the fastest ways to lose customer trust, so if I’m not confident I can maintain the image, I don’t add it.

But that approach might be too rigid and avoidant. What are the advantages and disadvantages to adding screenshots to documentation?

Why screenshots are bad

Screenshots in documentation can have a lot of problems:

Outdated screenshots can cause customers to lose trust in documentation accuracy.
Pages load more slowly when they have a lot of screenshots compared to pages that only have text.
Screenshots can replace step-by-step instructions, losing an explanation of why you might perform a task and replacing it with an image of how or where.
Screenshots often have no or poor alt text, making them inaccessible to those with low or no vision.
Relying on screenshots can lead you to write documentation that only describes the UI, instead of what you can do with it or what the data visible in a dashboard means and where it comes from.

However, these are mostly implementation problems. It’s possible to have screenshots in documentation without these problems.

Why screenshots are good

Screenshots and images in your documentation can serve some helpful functions:

Make it easier for readers to scan to a relevant part of the content.
Provide a visual frame of reference for people reading the documentation without using the product at the same time.
When used correctly, they can supplement confusing task steps.

Indeed, the Baymard Institute performed user research about SaaS products and found that 35% of SaaS Sites Fail to Make the Service’s UI Sufficiently Prominent to Prospects, making it harder for potential customers to learn more about the product. The findings apply to a product’s entire website presence (beyond the product documentation), but it’s important to keep in mind when deciding whether to add screenshots.

In light of those findings, it’s unfortunate that SaaS products typically update the UI constantly, making it especially hard to maintain accurate screenshots.

But these are some real advantages to providing screenshots in your documentation. So how do you add them while still avoiding the problems?

Identify the purpose of a screenshot

If you focus on the purpose of a screenshot in product documentation, you can identify a few situations where screenshots make sense:

Make a confusing page clearer by adding context to it.
Give someone a sense of how a page might look when it has data in it.
Walk users through a task that traverses multiple different pages in the UI.

In other cases, a screenshot might function as filler and not provide much of a function while still being difficult to maintain. Don’t add screenshots in these cases:

For every step in a task.
To provide visual interest without a reference in the text.

You also don’t want to provide screenshots to support text that describes every element on a specific page, because you don’t want to write documentation that only describes the user interface ¹.

Make screenshots accessible

Whenever you add a screenshot, make it accessible. Accessibility of images is relevant for lots of different people:

A data analyst with low vision that keeps the documentation site zoomed in at 400%.
A sales engineer using a mobile hotspot to work at the airport.
A sysadmin visiting family in a rural area that just got woken up by an urgent page in the middle of the night.
An engineer using a localized version of your product in their own language.

If critical content is only available by looking at an image, it isn’t accessible to any of those people. Therefore, make sure that a screenshot isn’t the only way to gather important information.

You can start improving image accessibility by writing useful alt text. Refer to Microsoft’s excellent guide, Everything you need to know to write effective alt text.

You also need to consider the file size of any image that you add to your documentation, because image size can affect page load. Compress your images using a tool like compressor.io (or others compiled by Smashing Magazine) before adding them to your documentation.

If you want, you can work with a translation service to localize images in your translated content as well, but you can also use other approaches to help make them accessible to folks who are reading your documentation other than the language your content is written in, such as by simplifying your screenshots.

Use simplified screenshots

Being intentional about adding screenshots to your documentation helps reduce the maintenance burden because you have fewer screenshots to maintain. But if you use simplified screenshots, you can reduce the maintenance burden further.

A simplified screenshot is a screenshot modified to obscure or remove content irrelevant to the purpose of the screenshot. Anton Bollen introduces the idea in his excellent article, Simplified graphics: Meet the new design style for technical communication.

For instance, you might provide a screenshot with some content blurred out, to help readers focus only on the relevant portion of the image.

While they might take a bit longer to make, simplified screenshots don’t need to be updated as frequently because you can obscure parts of the UI that are irrelevant for your task, unavailable to users with specific roles or permissions, or that might otherwise be confusing.

With a simplified screenshot, it becomes outdated at the same time as your text content, so you can update them both at the same time—rather than your image becoming outdated sooner because some other part of the interface changed.

Automate screenshots

If you’re committed to providing screenshots and have a robust docs-as-code pipeline, you can also consider automating screenshot creation and maintenance.

Simon Willison wrote and released a tool called shot-scraper which you can use to automate taking screenshots of webpages (and by extension, web applications). I haven’t used it, but I can envision designing a screenshot pipeline that works as follows:

Attach a hook to a UI component that you use in a screenshot.
If the code changes, prompt the tool to automatically take a screenshot in a demo environment.
Compress the resulting image.
Create a PR to add the new image to the documentation, replacing the old image.

Of course, componentized UI code means that this could be pretty difficult to instrument on a large scale (the only scale at which this would be necessary), but depending on your code base and technical skills, this might be feasible for your documentation.

Intentionally add images to your documentation

If you have a strategy and a process for adding and updating images in your documentation, you can avoid the pitfalls of screenshots in your docs and bask in the advantages.

Add screenshots to the documentation only when they’re relevant and useful.
Make screenshots accessible, so even if they aren’t there, the content is still useful.
Use simplified screenshots to ease the maintenance burden and increase the relevance.
Consider automating screenshots if you have the resources.

In rare cases, you might be writing UI reference content, such as when documenting critical workflows for a process where the UI has just had a major update and buttons have moved around, or if you’re documenting software for users that have never used software before. But otherwise, if you find yourself writing content that describes the available buttons on the UI, you might want to consider your audience more fully. ↩︎

Improving documentation findability in an age of low-quality search results

Tue, 05 Dec 2023 20:57:56 +0000

For months, there have been reports on deteriorating search quality ¹. As the quality of search results deteriorates, so too does an important factor that makes technical documentation so useful — its findability.

In an era of web-first authoring and product-led growth marketing strategies, organic documentation findability matters. For some software documentation sites, at least 80% of traffic to the site comes from organic search.

You don’t want to abandon all efforts at optimizing for search engine results ², but it might be time to reduce our reliance on search as the primary front for documentation findability.

If we stop expecting readers to discover documentation on their own using search, how do we as technical writers make sure the information can still be found? What’s next for documentation findability?

A common tactic to improve search result quality is to append “reddit.com” to get higher quality results. The results are higher quality because they’re human-curated and generated, rather than algorithmically surfaced ³.

Reddit is just one site with human-moderated and curated conversations — your community is also having discussions in Discord or Slack communities, in the comment thread of a Substack newsletter, in person at events, and more.

If you support and engage with your community by responding to documentation feedback and requests, your community members show up to these conversations and discussions with awareness of the documentation—not just their most-referenced pages, but recently updated and published ones as well.

Why does this matter? I always say that my “north star metric” of documentation success is “if someone asks a question and a link to the documentation can answer it”.

If you write high quality documentation, and build champions of your product (and your content), those champions bring your content to the product and business problem conversations happening across the community.

That sounds amazing — but how do you make sure those champions find your content?

If you document a product with a user interface, you likely have some amount of in-product help. Oftentimes, that can be pretty basic.

For example:

A tooltip to describe something.
A “learn more” link in an error message that goes to the documentation.
A sentence of explanatory text in a dialog box that links to the documentation.
A link to the documentation.
A search box in the product that lets you search the documentation.
A help command in a command-line interface with varying amounts of information available.

As interfaces get more complex and documentation content gets harder to find on the web, it’s beyond time to think beyond the tooltip about other ways to incorporate documentation into the interface in a helpful, maintainable fashion.

We’ve all encountered the overly aggressive product tour when using a new web app, “helpfully” highlighting different parts of the interface the very first time you open it, either distracting you from your intended task (and getting in the way) or showing you about new features when you’re not even sure what your task might be yet (just trying to explore).

Instead, some other modalities could help:

Interactive tutorials that use the product to explain it — example code or templates.
A “show me how” option that demonstrates a discrete task on demand, rather than a tour that appears on first page load.
Autocomplete for commands with in-line guidance about required arguments and parameters.
More robust CLI-based assistance.
An API parameter that provides usage details for the endpoint.
High quality error messages.

Not all of these modalities are new, but they are still easily neglected!

Having help available in these areas isn’t enough. If your command-line tool has a help interface, does it list the available arguments for a command with no context, or is it descriptive and useful? If your API endpoint returns error messages, how helpful are they in helping the recipient fix the problem?

It’s expensive to create and maintain these content types, but if your documentation team is already integrated with your design and engineering teams and processes (hint), then producing this content extends, rather than transforms the way technical content is produced at your company.

If your documentation team is focused on putting out the latest fire or picking up what was thrown over the wall, you might need to invest more deeply in what content and customer assistance looks like at your company to provide some of these in-product modalities.

The closer your content teams are to the product design and engineering process, the easier it is to enshrine consistent terminology usage and avoid mismatches with mental models.

These modalities aren’t just for “product growth” initiatives—they’re about customer enablement and support, helping customers use and learn more about how to do things with the product they pay for—just like the documentation does.

Embrace AI: Chatbots and more

In the spirit of meeting your customers (and prospective customers) where they are, you need to embrace AI.

By this, I don’t mean “train a chatbot on your documentation content and make it available to customers”, because that adds a new interface that customers need to discover, learn how to use, and which you then need to maintain ⁴.

Instead, focus on making your content easily consumable by a language model as training material. That might look like:

Write succinct and clear content.
Use consistent terminology.
Structure your content consistently.

While large language models don’t need to be trained on structured content, the token-based retrieval methods used to generate text rely on vector similarity and associations. If your content contains consistent patterns then it’s likely that the computed vectors for relevant content in your documentation are going to be more accurate.

If your content is used to perform retrieval-augmented generation when prompting a large language model to generate a response, if the chunks created as part of that process all contain relevant information, the results are more likely to be relevant as well ⁵.

For example, if your content is always structured with this pattern:

Enable updates for your device

To enable updates, you must have admin privileges on your device.

To enable updates, do the following:

Log in as an administrator.

Navigate to the system settings.

Select system > general > updates.

For enable updates, select the checkbox.

A language model prompted to retrieve information about the permissions required to enable updates can more easily do so than if your content is structured like this:

To adjust system settings, you must be an administrator.

The available system settings for your device include:

Network

Audio

Appearance

Privacy and Security

General

Battery and Power

It’s important to keep your device updated at all times. It protects you from security threats like malware or viruses. You can set this up automatically if you turn on the correct setting. You can also adjust other settings that can help with security and privacy.

You might notice that the less-structured example is also hard to read and confusing. I’m intentionally exaggerating this example, but the more clear and concise the content, the more helpful it is to your readers and to LLM tools performing retrieval-augmented generation.

Improve documentation findability by going beyond the page

Ultimately, the best way to make your documentation easier to find is to put it where your customers and prospects are looking for helpful information:

Support your community members sharing links to your documentation.
Share content inside product interfaces (web and otherwise).
Implement consistent language and patterns to help readers and LLMs retrieve relevant information.

Take your content beyond the webpage and stay helpful!

Reports such as: June 2022, The Open Secret of Google Search by Charlie Warzel for The Atlantic. May 2023, What happens when Google Search doesn’t have the answers? by Nilay Patel for The Verge. June 2023, A storefront for robots by Mia Sato for The Verge. August 2023, Google’s Search Box Changed the Meaning of Information by Elan Ullendorff for WIRED. ↩︎
Mostly because the best way to ensure high-quality search results for your documentation is to write high quality documentation that is user-centric and task-oriented, and therefore matches the tasks and keywords that readers are searching to find your content. ↩︎
To be pedantic, algorithms of course are involved in who sees what on Reddit, but the responses and posts in any given subreddit, if well-moderated, are from real people. ↩︎
If you want to create an LLM-based tool that uses your documentation content, consider how to make it findable. For example, you could create a GPT in the OpenAI marketplace that is enhanced with a vector database full of tokens from your documentation site and can answer questions about your product—and perhaps even perform tasks in your product. You could also enhance autocomplete in your product with documentation content that is retrieved using an LLM integration. Think beyond the chatbot. ↩︎
I’m making this assertion based on the blog post RAG Isn’t So Easy: Why LLM Apps are Challenging and How Unstructured Can Help, which is confusing to read because the company publishing it is called Unstructured. In the post, the author demonstrates how chunks created with their tool happen at semantically relevant spots, such as before a heading, thus finding that “The RAG system produces higher fidelity responses with Unstructured chunking because the chunks have more consistent semantic meaning, resulting in more relevant query results.” ↩︎

Why web design sucks now

Thu, 26 Oct 2023 22:05:43 +0000

Heather Buchel’s post It’s 2023, here is why your web design sucks about the current state of web design (and web app design) resonated with me, especially this quote:

“Design decisions can only be pushed so far to the left before we realize the system is broken”

If you bisect design and development into different professions, you can end up with designers that aren’t technical enough to design what’s possible, or developers without enough design prowess to make design decisions independently.

I wonder too if tooling has something to do with it — in an era of Figma and Canva and Webflow, it’s easier than ever to design websites and web applications without having to interact with HTML, CSS, or JavaScript, let alone the DOM of a web browser and poking around in DevTools to get something to display properly.

A similar issue can happen with technical writers in a “throw it over the wall” documentation culture, where a writer relies on functional specs, design docs, and engineering requirements documents to write content rather than actually using the product.

Part of it could be a hiring prioritization motive, where engineers are the first and most-valued hires, while designers and technical writers are seen as support staff and hiring is calculated according to team:designer or team:writer ratios, rather than a partnership-based approach.

In my opinion, cross-functional partnerships with empowered, collaborative product development, are crucial for building a high quality product that customers can use.

How to add documentation to your product life cycle

Wed, 06 Sep 2023 10:53:52 +0000

As a tech writer, I’ve encountered a number of different processes that teams and companies have used to add documentation to their product development processes. Some of these are intentional and others are incidental—but all are used to create documentation across the software development industry.

At its most basic, the product development life cycle involves the following steps:

When you add documentation, that might look something like this:

The typical models I’ve encountered working as a tech writer can be categorized as follows:

The “throw it over the wall” model

The “throw it over the wall” model happens when engineering and documentation teams don’t have much interaction. Instead, in a waterfall-like approach, the development team creates something and tells the documentation writer about it after it’s built.

This can happen intentionally or incidentally. Maybe an entire feature gets developed, is ready to launch, and someone lets the documentation writer know—before or after the release.

Advantages

The advantages to this model are largely to the organization:

Can work well across time zones where the engineering team is in one location and the writer is in another, because you don’t have to try to do synchronous work like attend the same meetings.
Faster to write documentation because the product or feature is fully functional when the writer starts drafting. Encountering bugs during the drafting process can create a feedback loop where the writer can help improve the product and maintain (or build) empathy for the reader, but it definitely slows down drafting when key functionality is broken or not developed yet.

Disadvantages

The disadvantages of this model are both to the customer experience:

Little to no ability to improve or change the product in response to documentation feedback, because development is already complete. At best, the writer can file bugs or stories to get picked up later if they are validated by customer feedback. // for the product
Inconsistent feature coverage in the documentation because only the items that get thrown over the wall as part of a request for documentation get documented. This can lead to content drift, where the docs no longer match the product in some cases, or are missing some features that could use documentation.
Delays between feature launch and published documentation. Depending on when the feature is thrown over the wall, you can get delays between launching the feature and publishing documentation. If that happens, customers might not find out about the new feature, or if they do, don’t have any documentation to help them if they’re confused or something is broken, forcing them to file a support case if they can. Ultimately, this delay can make customers dissatisfied with your product.

And to the organization and its writers:

Unpredictable workload for the writer. By not being involved in the planning process and finding out about a project when it lands on the other side of the wall, there’s a constant juggling of priorities with little flexibility on timelines. If every project appears as a surprise, it’s pretty difficult to plan resources or estimates.
Pressure to write documentation quickly because documentation was left as the last step in the process, they’re seen as a bottleneck to the release process.
Low context about the product and low rapport with the team. You might produce documentation that misses key functionality that wasn’t apparent to you when you encountered the functionality, or not have a full understanding of what was built or why, depending on what the handoff process looks like.

The “put out the latest fire” model

This is a fairly common model, especially for overburdened documentation writers or those working as contractors or consultants on a project. You could be an external freelancer brought in to work on a specific project or be part of a documentation or content team that works with others on a project-by-project basis.

In this model, you join a project and work with that project team in an embedded or throw-it-over-the-wall fashion until the doc needs are met. You might only have one point of contact, like a product manager, and you deliver the documentation requested by them. After the project concludes, you join a different project.

Advantages

The advantages to this model are largely to the organization:

Lets fewer tech writers work across more projects. Whether it’s successful is debatable, due to the volume of context switching required by the writers.
Faster onboarding onto new projects or teams, since the onus is on the project team to build the context for the writer, often to the degree of writing drafts or sending screenshots to the writer.
Can permit flexibility in commitments, where projects without allocations go intentionally undocumented (or get delayed) because the documentation team doesn’t have the resources to support a requested project, or you can move a writer from one area to another

Disadvantages

The disadvantages are borne by both customers and the organization:

Results in feature-driven documentation rather than user-centric documentation. Because the writer has less context on the what, why, and for whom of the project, you can end up with feature-driven documentation that describes what was built rather than what the reader can accomplish with the new functionality.
Estimating documentation workload is challenging due to the overlapping and intersecting projects, combined with the difficult-to-anticipate effects of context switching. More time can be spent juggling priorities than writing, and writers can quickly end up overburdened compared with their colleagues.
Needs org-wide process consistency to be successful. If project teams don’t work using the same (or at least clearly documented) processes and rituals, the onboarding process onto a new project or team can be jarring and slow, somewhat erasing the benefits of moving writers to other projects as they’re needed.
Docs are unowned and maintenance gets left behind. Because you leave a project as soon as you ship the documentation, there isn’t much of a concept of long-term ownership, just whoever worked on something last. Without ownership, maintenance of the documentation content is ignored in favor of the latest new feature work.

The “I’m with the team” model

In this model, the documentation writer is treated as a full-fledged member of the engineering team, involved in all the same meetings, Slack channels, and most discussions.

Advantages

The advantages of this approach are mostly for the customer:

User-centric documentation rather than feature-driven documentation. If you know what is being built, why, and who for, it’s a lot easier to craft documentation that is guided by the user goals guiding product development rather than writing down what was built and how it works.
Increased consistency in product text (and improved product quality). Because the writer is involved throughout the development and design processes, they can contribute to text in the product—whether UI text (content design) or API specifications and descriptions.
Accelerated content development. With ultimate context and involvement on what is going on, and little if any context switching, the writer can produce documentation fairly quickly and without too much delay before and after development and testing ends.

Disadvantages

The disadvantages of this approach are borne largely by the organization:

It can be resource intensive. It’s often not economical to have a 1:1 writer to team ratio because not every project that engineering works on requires documentation. Of course, if engineers are working on technical debt, writers can also spend time on documentation debt or professional development, but this sort of luxurious level of staffing isn’t one that I’ve ever seen. Depending on the pace of development, the optimal writer ratio might be closer to 1:3 writer to teams, but that’s still higher than the other models.
Writers can waste effort and get confused if they start drafting before the product is finished “enough” to start drafting. If you document every iteration of a feature flow before it is finalized, you might write the same procedure multiple times. Sometimes these drafts let you get to the most efficient and concise way of explaining something, but there’s also a risk that a writer gets attached to an incorrect mental model and the procedure ends up less clear than it might have otherwise.
Can introduce single points of failure. If every writer is embedded in a specific team, there is a risk that if that writer goes on vacation or quits, no one else will have enough context to cover for them effectively.
The writers can be siloed and lack visibility into other projects at the company. Because the writers spend more time with their engineering team(s) than the documentation team, this approach takes extra effort to build documentation team cohesion and consistency across the writing styles on the team.

Which model is the best?

It depends on your priorities and circumstances!

In an ideal world, I think that the “I’m with the team” model is the one most likely to produce the best documentation — but it’s also the most resource-intensive, and therefore might not be the most feasible option for every organization.

The “put out the latest fire” model can work well at a company with low-complexity projects and consistent organization-wide procedures, making it possible to plug-and-play writers on demand to different projects. Unfortunately, many projects that require documentation also benefit from strong domain expertise and local project context, so this approach is best suited for short term transition phases at a company. After you know what projects are going to stick around and need writing commitment, you can invest in a more proactive and embedded approach like the “I’m with the team” model.

For small companies like a seed-stage startup, or one that is great at asynchronous communication, the “throw it over the wall” approach lets you add documentation without changing your core engineering processes. If you’re at the early stage where you’re prioritizing moving as fast as possible most of the time, this approach probably makes sense.

Documenting machine learning models

Thu, 06 Apr 2023 10:46:19 +0000

Products use machine learning and “artificial intelligence” to do things like recommend a song to listen to, offer a quick response to an email, organize search results, provide a transcript of a meeting, and more. Some products rely on ordinary data analysis to construct insights about things like your business performance in a market, or the conversion rates for your online shopping site.

Unfortunately, the companies providing these products often gloss over the technical details of how these systems work—making it seem like those tools are magic, omniscient, or just plain inscrutable. Algorithms are so common, yet it’s just as common for the logic programmed into the systems to be hand waved away.

As integration of machine learning, artificial intelligence, algorithms, and data analysis becomes standard and expected in software products, internal and external documentation for those systems must also become standard.

As a technical writer, it’s perhaps not surprising that I think documentation is important.

Writing documentation is a way to communicate information about your product—and that information in turn lets others use and understand your product.

Commenting code, providing READMEs, writing how-to guides — all of these forms of documentation help people understand and interpret your code, evaluate your project, and use your product.

The normalization of data-driven software, where machine learning models drive key product functionality, means that folks in operations, procurement, legal, and more departments need to understand the components of the product, how it might interact with their system and users, and what risks the business might face as a result.

That makes it extremely important to document each component of a data-driven system, from the datasets involved in data analysis and machine learning model training, to the machine learning models and model results, as well as the systems in which the machine learning models exist in.

When evaluating software, product documentation is expected. It would require an immense amount of trust, and likely some promises, to buy or even start using a free software product that lacks documentation. It’s also much faster and easier to start using software if it’s well-documented.

But machine learning models and datasets available on the web are often not well-documented.

Sometimes that can happen because you encounter a machine learning model without realizing it. You’re listening to music and a model is queuing up the next recommended track in Spotify’s autoplay, or you’re browsing the web and you read an article generated by a large language model. Other times, you’re more aware that you’re interacting with a model, such as when you feed a prompt into Midjourney or Stable Diffusion, or ask ChatGPT a question.

Much like Apple might want you to feel when using their products, artificial intelligence aficionados want you to be able to use their trained models without the friction of documentation about how it works.

But documentation can be vital to make AI-based products more useful. For tools based on generative models, like Midjourney or ChatGPT, understanding what led to the output that you see can help you tweak your input to produce more useful results.

This holds true for data analysis and other machine learning implementations as well, such as any data analysis project in an organization, or internally deployed machine learning models supporting business processes and more. Documentation can provide a lot of support.

Randy Au makes the case for documenting a data analysis process in his post Data Science Practice 101: Always Leave An Analysis Paper Trail:

Despite how much busy work it sounds like right now, you need to leave a paper trail that can clearly be traced all the way to raw data. Inevitably, someone will want you to re-run an analysis done 6 months ago so that they can update a report. Or someone will reach out and ask you if your specific definition of “active user” happened to include people who wore green hats. Unless you have a perfect memory, you won’t remember the details and have to go search for the answer.

Analysis deliverables are often separated from the things used to generate it. Results are sent out in slide decks, a dashboard on a TV screen, or a chart pasted into an email, a single slide in a joint presentation for executives, or just random CSV dumps floating around in a file structure somewhere. The coupling between deliverable and source is non-existent unless we deliberately do something about it.

Without a clear definition of how an analysis was performed, how data points were defined, or where a particular chart came from, the results lose value, credibility, and reproducibility. It helps the future version of you, as well as folks that you collaborate with, if you document the context of a project and store that documentation alongside the output of the project.

Similarly, without a clear sense for how a model was trained or why it might be producing specific results, the results lose value and credibility. In an era where generative machine learning models output fabricated academic references when you ask it for citations about a topic, documentation about the systems becomes even more valuable to help you judge which machine learning model output you can trust.

You can more accurately assess if the outcomes delivered by your model are valid, or due to making the error of testing on your training data, if you have clear documentation about how the model was trained and on which datasets.

As Emily Bender points out in the Radical AI podcast episode, The Limitations of ChatGPT with Emily M. Bender and Casey Fiesler:

If you don’t know what’s in the training data, then you’re not positioned to decide if you can safely deploy the thing.

Especially if you’re using a machine learning model in a commercial context, you want to be able to identify possible harm that could arise from the output, such as violent, misogynistic, racist, or other dangerous output. Without information about the training datasets, model training practices, and overall system context for a model, you can’t properly evaluate it.

Provide helpful documentation

What does it mean to provide helpful documentation for the components of machine learning and data analysis practices?

As with any documentation, you need to consider the following:

Who is the audience?

The audience of the documentation depends on the context of the analysis or the model. The documentation for a dataset distributed on the web for free on Kaggle has a different audience than a machine learning model deployed internally at a large financial institution to detect fraudulent credit card transactions. As such, the content of your documentation might change accordingly to include more or less detail about specific aspects of the data.

What purpose does the documentation serve?

Similarly, the purpose of the documentation is different for a publicly available dataset when compared to an internally deployed machine learning model, and is different still from a product that provides a chat interface for a large language model. The documentation for a machine learning model used by a financial services company needs to exist for regulatory and auditing purposes, in addition to the typical purpose of remembering how the model works.

How do you provide the documentation?

How you provide the documentation also differs depending on what you need to document. You might be able to add inline comments to a dataset, but then you don’t have a good way to provide an overview of the dataset itself. A machine learning model offers no simple way to include documentation as part of the training output. So far, most standards involve providing a PDF with information, but others such as Hugging Face Dataset Cards, implement a YAML-formatted specification file to publish alongside the dataset on the Hugging Face Hub.

What do you put in the documentation?

When it comes to determining what the documentation should contain, it depends on which component of the machine learning process you’re documenting, and there are a number of standards proposed by academics and prominent players in the data industry.

How to document machine learning components

If you’re deploying machine learning models in your business operations, disseminating the results of data analysis, or integrating machine learning into your product, you need to write documentation. What you write depends on the part of the system that you choose to document.

There are several perspectives to consider:

documenting the datasets
documenting the models
documenting the model results, or output
documenting the system

Documenting the datasets

When you focus on documenting the datasets, you want to capture things like:

How recently the data was collected
From where (digitally and geographically)
How representative the data is against various parameters
Why the dataset was created

For labeled datasets, you want to consider additional components:

Which annotation process was used
How much data was annotated
Which measures you used to validate the annotations
Who annotated the data

Standards for documenting datasets:

Data Cards from Google, which aim to

“provide structured summaries of ML datasets with explanations of processes and rationale that shape the data and describe how the data may be used to train or evaluate models.”
Datasheets for Datasets from Microsoft, which provides a framework of

“questions about dataset motivation, composition, collection, pre-processing, labeling, intended uses, distribution, and maintenance. Crucially, and unlike other tools for meta-data extraction, datasheets are not automated, but are intended to capture information known only to the dataset creators and often lost or forgotten over time.”
Data Statements from University of Washington:, which offers

“a characterization of a dataset that provides context to allow developers and users to better understand how experimental results might generalize, how software might be appropriately deployed, and what biases might be reflected in systems built on the software.”
Data Biography from We All Count, which you can use to record

“a comprehensive background of the conception, birth and life of any dataset … an essential step along the path to equity in data science.”
Dataset Cards from Hugging Face, which

“help users understand the contents of the dataset and give context for how the dataset should be used.”
Dataset Nutrition Labels from MIT and Harvard University:, which offers a

“diagnostic framework that lowers the barrier to standardized data analysis by providing a distilled yet comprehensive overview of dataset “ingredients” before AI model development.”

Documenting the models

In addition to documenting the datasets used for data analysis or machine learning training, you also need to document the models trained on the datasets.

When documenting a machine learning model, you want to capture things like the following:

What data was used to train the model
How the model was trained
How the model features were tuned
Why the model was created
How the model output was tested

For versioned machine learning models, it’s also helpful to include context about what is different between one version of the model and the previous, such as:

Why the new version of the model was created
What training data is different between this version and the previous version

Standards for documenting machine learning models:

Model Cards from Google, which

“provide practical information about models’ performance and limitations in order to help developers make better decisions about what models to use for what purpose and how to deploy them responsibly.”
Data Portraits from Johns Hopkins University, which provide

“artifacts that record training data and allow for downstream inspection”, making it easier for model evaluators to perform tasks like determining if an example output was part of the data input for a model.
Reward Reports from University of California, Berkeley, which are specific to reinforcement learning models and provide a framework for the intended behavior and reinforcement tactics employed for a given reinforcement learning model.

Documenting the model results

It’s important to document the results produced by a model in specific scenarios, which can help you debug and retrain the model as needed. Documentation about model results is often referred to as “explainable AI”, as the goal is to explain the outcomes produced by artificial intelligence (one or more machine learning models, or algorithms).

When you document the model results, you want to collect the following information:

The dataset used to train the machine learning model (and the documentation for the dataset)
The version of the machine learning model (and the documentation for the model, such as the model features)
The input into the trained model
The output of the trained model, in response to the input
The expected output of the trained model, based on the input
How close the expected output was to the actual output, based on the input
Key metrics such as precision, recall, and F1 scores for the model results, especially across different feature segments.
Some possible reasons about why the model produced that particular output

Standards for documenting machine learning model results:

Method Cards from Meta, which

“aim to support developers at multiple stages of the model-development process such as training, testing, and debugging” with prescriptive “instructions on how to develop and deploy a solution or how to handle unexpected situations”
Four Principles of Explainable Artificial Intelligence from NIST, which proposes

“that explainable AI systems deliver accompanying evidence or reasons for outcomes and processes; provide explanations that are understandable to individual users; provide explanations that correctly reflect the system’s process for generating the output; and that a system only operates under conditions for which it was designed and when it reaches sufficient confidence in its output.”
AI Explainability Toolkit from IBM, which provides an extensible toolkit of options to explain AI depending on the audience of the explanation, such as

“data vs. model, directly interpretable vs. post hoc explanation, local vs. global, static vs. interactive”
Standard for XAI from IEEE, which

“defines mandatory and optional requirements and constraints that need to be satisfied for an AI method, algorithm, application or system to be recognized as explainable.”
EUCA: End-User-Centered Explainable AI Framework from Simon Fraser University, which offers

“a prototyping tool to design explainable artificial intelligence for non-technical end-users.”

Documenting the system

It’s important to document the entire system in which a machine learning model operates. A machine learning model is never implemented as a discrete object. Models must be kept updated to avoid drift and thus are implemented as part of a larger system that can include data quality tools, a testing framework, a build and deploy framework, and even other models, such as in the context of an ensemble model.

Documentation about the entire system offers helpful guidance to folks trying to understand a system so that they can maintain, update, debug, and audit the system, to name a few common tasks.

As such, machine learning system documentation needs to include the following:

Details about which tools exist in the system
Details about how updates to the system are performed
Dependencies within the system, including people
Data flows within the system
For models within the system, documentation for those models

Standards for documenting machine learning systems:

System Cards from Meta, to

“provide insight into an AI system’s underlying architecture and help better explain how the AI operates.”
FactSheets from IBM, which provide

“a collection of relevant information (facts) about the creation and deployment of an AI model or service. Facts could range from information about the purpose and criticality of the model, measured characteristics of the dataset, model, or service, or actions taken during the creation and deployment process of the model or service.” with the objective of allowing “a consumer of the model to determine if it is appropriate for their situation.”
DAG Cards from Metaflow, which offer

“a form of documentation encompassing the tenets of a data-centric point of view. We argue that Machine Learning pipelines (rather than models) are the most appropriate level of documentation for many practical use cases, and we share with the community an open implementation to generate cards from code.”

Actually write the documentation

Making sure the documentation actually gets written is the most important aspect of documenting machine learning components and systems.

You can choose to automate portions of the documentation, identify different points of the model development process where it would be prudent to update part of the template that you choose, or whatever works for your team and your processes. You can also define robust accountability mechanisms like checklists.

If you lack processes and accountability, it’s easy to skip writing documentation. All the standards in the world for documenting data-driven systems don’t matter if the documentation never gets written.

Most fields struggle to incorporate documentation into their processes, and build accountability for making sure it gets written, but fast-moving fields like data science and machine learning that are still formalizing their practices struggle even more.

No matter which method you choose for documenting data-driven systems, you must include writing the documentation in your existing workflows.

Chat apps are no substitute for documentation

Wed, 15 Mar 2023 09:25:14 +0000

As a technical writer, I have a mixed opinion of chat applications like Discord and Slack. On the one hand, they make it easy to quickly get ahold of someone who can answer your questions, which is a relief if you’re struggling to gather information you need to write a draft.

On the other hand, because it’s easy to quickly get ahold of someone who can answer your questions, that convenience can implicitly incentivize folks to neglect documentation. This is true on both sides:

Folks with questions might ignore documentation that exists because it seems easier or faster to just ask in the chat app.
Folks answering questions might prefer not to spend lots of time writing system design documentation, detailing the decisions they made and why, when they can just answer those questions as needed.

Jason Scott shares this mixed opinion. As he points out in Discord, or the Death of Lore:

“I have no disputes as the popularity of the places, the things that happen there, and the unquestioned vivaciousness of being the party that never seems to end and everyone wants to join.

I just happen to be the sort of person who notices there’s no decent fire exits and most of the structure is wood and there’s an… awful lot of pyrotechnics being set off.”

I feel the same way.

I wonder if the folks who have their focus time interrupted by detailed, somewhat archaic questions that require a backstory, ever wish they had documented the answer so that they could respond with a link and move on with their day.

Chat apps like Discord end up diluting the available knowledge because the content shared in them isn’t persistent, and the allure of an always-available answer breaks down when the person that could answer is no longer available. Jason refers to this as the “lore-to-knowledge transfer”:

The danger in this process, the potential lost ballast in the rise to the skies, is that the lore-to-knowledge transfer is lossy, messy, and arbitrary. Maybe those in the know want to keep the information to themselves, so it won’t be given to whoever the person or persons are who are laying down the written form. Maybe the chronicler of information has blind spots they don’t know about and not enough people to correct them. Or, more likely, you have to set the “noise filter” of the information to not go down the rabbit and rat holes of contingencies that maybe a dozen or two people will even want to know about, to the favor of that which everyone will need. The outcome is always the same: Lore loses in the long run.

The process of documenting information can break down silos (information is more available), expose blind spots in an approach (something is missing here), and enable asynchronous knowledge transfer (documentation is always online).

Etsy blogged about an interesting approach five years ago, Etsy’s experiment with immutable documentation, where they built a plugin that engineers could use within Slack to update or create documentation:

At Etsy we’ve developed a system for adding how-docs directly from Slack. It’s called “FYI”. The purpose of FYI is to make documenting tactical details – commands to run, syntax details, little helpful tidbits – as frictionless as possible.

I’d be curious to learn whether they’re still using this system, and how it has aged.

Where to start with analytics for documentation

Sun, 27 Nov 2022 10:58:56 +0000

It’s tough to find helpful information about analyzing website metrics for technical documentation sites.

The goals of technical documentation are different from those of a marketing blog or a company website. You’re not optimizing for maximum traffic. No one is clicking “Add to Cart” on your API reference topics.

You need to use slightly different metrics and in different ways than you might for a marketing blog or your company’s website.

If you want to start using site analytics with your documentation but aren’t sure where to start, this post is for you.

What are site analytics?

Website analytics, in the context of this post, are the metrics collected about a user’s activity when they access a website—in this context, the website hosting your product documentation.

Those user metrics are then aggregated and made available as website analytics such as page views, unique users, average session length, page referrer, and more.

Typically, companies add analytics when they launch their websites, so your documentation site almost certainly has a tool like Google Analytics, Adobe Analytics, or a smaller provider like Hotjar set up.

Get a basic understanding of working with site analytics

Start with these blog posts from Bob Watson:

Those blog posts are an excellent foundation for building a basic understanding of what site analytics mean.

Make sure to get familiar with the tool that you’re using to analyze the data¹. If you’re focusing on one section of your documentation, learn how to filter the data so that you only see relevant values².

You also want to look at site analytics with several months worth of data. For example, as Kumar Dhanagopal points out in his talk for Write the Docs Portland 2022, Don’t trust the numbers!, it’s more valuable to look at trends rather than absolute numbers.

With that in mind, don’t look at analytics for new pages until a few months have passed, to make sure the metrics are relevant and reflect a consistent pattern.

Start looking at your site analytics

As tech writers, we’re frequently understaffed and overburdened. There’s always more writing to be done than we can tackle. Because of that, we need to be strategic.

Data can help when we’re trying to figure out how to do more work with our time.

Maybe these scenarios sound familiar:

I have a lot of outdated content that I need to update, but where do I start?
People keep asking questions in Slack and in forums that are answered in the documentation, but why can’t they find what they’re looking for?
My boss wants to put all the documentation inside the product, but I’m not so sure.

Site analytics can help with these scenarios if you can ask specific, discrete, and data-focused questions of the data. In this way, website analytics function as imperfect proxies³ for the information that you actually want to know.

Define and refine your questions

When you initially come up with a question, it might not be something you can answer with data. What you can do is refine what you want to know into a more data-focused question, and then identify which website analytics metrics might help you answer it.

What you want to know	Data-focused question	Website analytics metric
What do people find useful?	Which pages are most popular?	Page views
Where should I start updating outdated content?	Which pages are most viewed?	Page views
Is the documentation easy to find?	How are people getting to the documentation?	Channel, referrer, and source
What do people want to know about the product?	What searches are leading people to my documentation?	Search terms
What portion of our overall user base uses the documentation?	How many users are viewing these pages?	Total users
Are people finding the information they need? Are they lost?	Are people clicking links and engaging with the documentation?	Click-through and engagement data

Although these are data-focused questions, data is rarely, if ever, definitive. You can’t expect an obvious answer to any questions you ask of the data.

Instead, your goal is to reduce your uncertainty about the answer. When you ask a question about your documentation, such as “How are people getting to the documentation?”, you can make assumptions about the answers.

Maybe people are searching, using a browser bookmark, or opened a link from an email or chat message.

After you make assumptions, you can look at the data to reduce your uncertainty and validate or reject your assumptions.

Which pages are most popular or most viewed?

If you want to know which pages are most popular or most viewed in your documentation, look at page views. Page views are exactly that — when someone views your page in a web browser.

How to look at page views

Look at page views across different time scales, to identify which pages are consistently viewed, and which ones are frequently viewed.

If you look at the page views for a month, look at the previous month, and the same month last year, to help you get a sense of baseline page views.

As Bob Watson points out in How you can make sense of your site analytics, it’s important to understand the baseline number of page views for your documentation site, and use that baseline to evaluate outliers for the pages that you want to learn more about.

I prefer to use a bar chart or a table to look at page view data so that I can actually figure out what I’m looking at. Bar charts help me identify outliers at a glance, and tables give me easier to read data.

Ultimately, you’re checking the data against your expectations and a baseline.

Evaluate against your expectations

You might see page views for a certain page spike during a week. This is when you start to add context and consider the possible causes of that spike:

You could look at page referrers to see if your page is being shared somewhere like Hacker News or Reddit and going viral for some reason.
You could also consider the product marketing and release schedule—if it’s a page related to a newly announced feature, it makes sense that the traffic would be higher than usual!

Beyond patterns in page views, I’d expect that the homepage for the documentation and a “quickstart” or “installation” topic are going to be the most-viewed and most-consistently viewed pages of your documentation. This might be true for you as well, but this is something else you can explore when you contextualize the data.

How are people getting to the documentation?

If you want to know how people are getting to the documentation, there are several metrics that give you insight into this at different levels of granularity.

If you want to know which social sites, search sites, or other websites users are using to get to your documentation site, session source has that information.

The page referrer helps you find out what path to and through the documentation users are taking, if any.

What metrics to look at

Channel data gives you information about the method that someone used to get to your documentation site. The following channels⁴ are common for documentation sites:

Direct
Organic search
Referral

Session source is the disaggregated form of the default channel grouping. Looking at the session source combined with the medium lets you see both the channel and the specific site. For example:

session source	medium
search	google
search	bing
referral	t.co

Use the page referrer metric if you want to know exactly which page sent users to your documentation. If you’re using Google Analytics 4, you’ll need to do some configuration⁵ to use it in reports.

Look for surprises

When you’re looking at referrer data (whether it’s channel, session source, or page referrer), keep an eye out for surprising or unexpected sources of traffic.

You might discover that a company confluence page shows up as a frequent page referrer to your documentation. As part of contextualizing the data, you can find out whether that company is a customer of your product.

If they’re a customer, you might want to find a way to reach out to them for a user research interview.
If they’re not a customer, you might want to dig into what their company does to learn more about why they might be linking so frequently to your documentation.

What searches are leading people to the documentation?

If you want to know which search terms lead people to the documentation, use search term data. It might seem obvious that search terms are a valuable source of information, but there is some nuance in terms of which search terms are available to you.

What search term data to look at

The search terms collected by Google Analytics and similar tools are the terms used to search your documentation site while on your organization’s website. Search terms in Google Analytics are not search terms that users are typing into Google⁶.

Instead, to see search terms that users are typing into Google that might lead to your website, you need to use the Google Search Console. You can see up to 1000 search terms for which your pages made an appearance in Google Search results. If you spend some time and effort customizing Google Search Console, you can see more helpful data⁷.

What Google Search Console can’t offer you, besides the search terms that people use on other sites like Bing, Duck Duck Go, Baidu, or Yandex, is search terms in context with a user journey.

The search terms available in your analytics tool can be more useful because you can identify which terms led people to specific pages.

What to consider when using search term data

Search terms are often more valuable for large documentation sets or products with a large user base, for the following reasons:

The larger a documentation set, the harder it is to find information by browsing a table of contents, so more people will use search.
The larger the user base, the more people will use search because you have more people using your website in general.

Because search terms are a proxy for what your users want to know more about, you can draw more relevant conclusions (or more confidently validate assumptions) when you have a high volume of search terms.

If your documentation set is about 10-30 pages, or you have a small user base for your product, you might not have many search terms, or enough search term traffic, to give you helpful information in search term data.

Evaluate your search results data

When you evaluate your search results data, whether in analytics or in Google Search Console, you want to look for the following:

What are the most consistently searched terms over time?
What are the most frequently searched terms?
What are the least-searched terms?
What are the inconsistently searched terms over time?

If you have enough search term data, you can start to contextualize the data with other types of information.

You can identify successful searches, where a search term led users to a relevant documentation page, and unsuccessful searches, where a search term led to no engagement or led users to an irrelevant documentation page.

If you have a high volume of unsuccessful searches for a specific term or related group of terms, that might indicate an opportunity to write new documentation or recontextualize existing documentation to use similar terms that people are using to search.

A lack of terms in search that you might expect to see could indicate that users are finding content a different way — by navigating directly to it using bookmarks, or using the on-page navigation and landing pages rather than using search.

How many users are viewing these pages?

To find out how many users are viewing the documentation, review the total and active users⁸ for your site. I’m intentionally referring to users viewing rather than reading the documentation, because we can’t know what anyone is doing when they’re on your page⁹.

While real time users is a metric calculated by Google Analytics, it’s not valuable as a documentation metric. Your documentation site is not an ecommerce platform attempting to convert viewers into purchasers or a SaaS app attempting to measure user activity for monetization purposes.

A high number of active users might indicate useful documentation because people are returning to it consistently. However, you can’t know why users are returning to the documentation without talking to them.

I find it more valuable to focus on total users and active users to get a sense for how many of your product customers are using the documentation when you contextualize the data.

Are people clicking links and engaging with the documentation?

If you want to better understand whether people are clicking links and engaging with the documentation, you can look at click-through and engagement data.

What engagement metrics to look at

Identify how people are engaging with your documentation with the following engagement metrics:

Link URL tells you which links people clicked in your documentation, and how many times.
Bounce rate tells you the percentage of sessions in which a user visited a page in your documentation and left that page within 10 seconds without doing anything.
Average engagement time tells you the average amount of time, for a given time period, that someone viewed your documentation.

These definitions and names can vary by analytics tool, but the meanings are roughly the same.

Evaluate your engagement metrics

To evaluate these metrics, look for outliers in activity and patterns, and consider your assumptions about user behavior.

When you look for outliers, you might look for the following:

High numbers of clicks on specific links compare to other links.
High time on page or engagement time compared to other pages.
A high bounce rate for a page.

You also want to contextualize these outliers in terms of frequency and volume. If one or two people are bouncing off a page out of a total user cohort of 10K, that’s likely meaningless. If thousands of people are bouncing off one page in the same total user population, there’s probably something worth investigating.

Recognizing patterns in the data that can indicate how people are behaving can also be useful. For example, if you created a supertask topic that contains links to 6 steps in a complicated decision tree process, you could gather the clickthrough data for that topic and see which links are being clicked most frequently, and from which page.

Relatedly, if you have a wayfinding navigation topic, you could gather the clickthrough data for that topic and determine if people are using the links in that topic at all, or not. If not, they might be using search or the table of contents instead of the links in that topic. You need to add context to reliably validate your interpretations.

Maybe you have a page in your documentation that introduces the API for your product, then links to an auto-generated API reference site. You can check your metrics to determine whether, as you might expect, that link gets a lot of clicks.

For most engagement data, you want to contextualize the data. You need to know information about the documentation to make assumptions about whether a low engagement rate on a page is meaningful. If it’s a landing page, maybe you want a low average engagement time, while you might expect a higher average engagement time on a reference page with a lot of function descriptions.

Engagement metrics are ultimately limited in their usefulness. They can’t communicate people’s motivations, so you can’t draw reliable conclusions about why people are clicking links, what’s contributing to a bounce rate, or provide a reason for a high or low engagement time. Engagement isn’t a straightforward proxy for reading.

If you really want to know more about people’s behavior and motivations, supplement these metrics with added context and user interviews.

Contextualize the data

When you interpret your analytics to try to validate your assumptions about the answers to specific questions, you need a thorough understanding of the data values and the likely causes.

It’s crucial to add context to your documentation site analytics to help identify the causes of unexpected data points, such as outliers.

With context, you can try to understand why your page views spiked on December 15th, or the email channel became a dominant referrer for a week in June.

When it comes to documentation site analytics, I find it most helpful to add the following types of context:

Add behavior context

Talk to users to add context to user behavior.

Beyond how many people are viewing the documentation, you also want to know why they are viewing the documentation.

Talking to users is the only way to answer questions like “why are people staying on the SDK function reference page so long?” or “how do you typically get to the documentation and why?”.

I always recommend supplementing site analytics with user interviews, as Bob Watson also recommends on his blog post, Tips for conducting documentation research on the cheap.

If you have access to a user experience researcher, work with them to conduct some high quality user research about your documentation. The Write the Docs website has helpful resources about user research.

Ultimately, the best way to know if your documentation is useful and valuable is to ask the people you’re writing it for.

Add product context

Add product telemetry and information to add context to your website metrics. It’s difficult to evaluate a baseline or understand patterns in your documentation metrics without an understanding of user behavior in the product.

You can do this in a number of ways.

Correlate with product telemetry

If you can, I recommend correlating site analytics with product telemetry data. Very few folks have an end-to-end data surveillance apparatus to follow users from web application to documentation site with one session cookie.

Instead of a complex approach, stick to the basics. Compare rough numbers of site users to rough numbers of product users. It’s only helpful to do this at a large enough scale to where rounding and measurement errors won’t affect any conclusions that you draw.

This context can help you evaluate whether a higher-than-average number of views for a documentation topic about a particular product feature is happening at the same time that the product feature itself is seeing a higher-than-average number of active users.

Beyond active users, you can talk to product managers and other internal stakeholders, to get an understanding of what product-related metrics and information they use. Some of it might be helpful to you.

Consider the product functionality

In addition to product telemetry, consider what you know about the product itself.

You’re likely to see more page views (and feedback) for topics about popular existing product functionality.

Maybe your product uses in-app documentation for some functionality, such as auto-complete for commands. If you see lower-than-expected page views for those pages, the in-app documentation could explain why.

As another example you might see frequent searches for a specific phrase that isn’t in the documentation, but in reviewing the product itself, discover that the product uses that phrase in the user interface. That’s valuable context to explain why customers are using that search term to look for documentation.

Add documentation context

Perhaps the easiest context for tech writers to add when reviewing site analytics is context about the documentation itself.

Consider topic type

You always want to consider the type of content you’re evaluating:

Is it task content?
Reference content?
Concept content?
Intro content?
A combination of all of the above?

Topic type matters because users behave differently on different types of content pages. For example, readers might look at a reference topic briefly after locating the information they need, but spend awhile on a concept topic reading and making sense of the information.

If your documentation isn’t strongly typed, you might not notice differences in clickthrough rates or time on page that correlates with topic type.

Consider content goals

You also want to consider the goals of that content, or the purpose of the page, to inform your expectations about how a user might interact with it.

If you’re writing in-depth content like a tutorial or an interactive scenario, you might expect someone to stay on those pages for a long period of time and click links only to other parts of the tutorial.

topic path	average engagement time	page_views	link URL	click_events
/docs/user-getting-started	02:01	1436	https://example.com/docs/user-next-steps.html	1201

On the other hand, if you’re writing reference content like a command or function reference, you might expect someone to quickly bounce off the page after finding the name of the command they couldn’t remember. Alternately, the page might have a very long time on page from a developer leaving the SDK reference page open while they write a script.

topic path	average engagement time	page_views
/docs/command-ref-init	00:20	554
/docs/command-ref-all-transforms	00:55	1203
/docs/sdk-function-ref	3:07	387

For a page that you wrote as a navigation or landing page to introduce functionality, you might want to check to see if that page is a common entry point for the documentation, and whether people are clicking the links on that page to get to subtopics.

topic path	page_views	link URL	click_events
/docs/get-started	3409	https://example.com/download	1807
/docs/get-started	3409	https://example.com/docs/user-getting-started	1436
/docs/get-started	3409	https://example.com/docs/install	1208
/docs/get-started	3409	https://example.com/docs/develop-start	604

Many users find information by searching, but there are still users who browse the navigation of websites to find what they’re looking for¹⁰. In that case, if more people are landing on subsections instead of the navigation page, you might conclude that more searchers than browsers are using your website.

Consider content scope

Beyond content type and goals, you also want to consider the scope of the content on a page.

A page that documents functionality that’s only relevant or visible to a small portion of customers is a page that you would expect to have a lower number of page views, or perhaps low engagement.

The methods that people use to get to a page might vary for alpha, beta, or preview content compared to content about a generally available feature. You might expect to see more direct links for content that documents early-stage features, as sales people share links directly with customers. Content about a generally available feature is likely found most frequently from search.

Consider content age

New content, or newly updated content might have a different page view profile than other pages. It might not get much attention because it isn’t indexed fully by search engines yet, or it might get a lot of attention depending on the business and industry context of the feature covered by the page.

When you publish a new documentation page or a new content rearchitecture, avoid looking at the metrics immediately after you publish. Leave time for the page views to settle into a consistent pattern before drawing in-depth conclusions from the data.

Older content that hasn’t been updated in some time might not get a lot of page views. It might get a lot because maybe it’s the canonical topic for installing your product, and the installation process hasn’t changed in months. Content age alone isn’t enough of a signal.

Add business and industry context

You also need to add business and industry context to your analytics to make sense of patterns and outliers that you identify.

Consider what your company is doing, and how that might affect your documentation site analytics:

Company behavior	Possible analytics pattern
Marketing department starts a new campaign with specific terminology	Increase in search terms with the same terminology
Company hosts a three-day conference	Higher page views before, during, and after the conference for new feature pages. Lower than average engagement during the conference
Leadership interviewed on a popular podcast or television show	Higher page views on the documentation homepage. Novel search terms or higher volume for basic search terms.

You also want to consider what is happening in the industry around your company or organization:

If your product was included favorably in an analyst report, such as the Gartner magic quadrant, page views and searches for your documentation might increase.
If your organization has been featured on a popular website or email newsletter, you might see a new channel or session source on your documentation site.

You can add helpful context to the metrics you’re trying to analyze with a good understanding of the business and industry context for your product and documentation.

Conclusion

It’s tough to know where to start using website analytics with documentation, especially when the tools and resources primarily address different use cases.

Analytics are intimidating, but remember, they’re fuzzy too. Fuzzy and imperfect proxies for what we really want to know: Is our documentation helping?

Despite being imperfect proxies, it’s useful to look at site analytics to reduce our uncertainty about the answers to specific, discrete, and data-focused questions.

After reading this post, I hope you can more confidently look at site analytics data, evaluate it against your assumptions, and add valuable context about user behavior, product usage, documentation types and goals, as well as business and industry activity.

What next?

Watch Kumar Dhanagopal’s presentation at Write the Docs Portland 2022, Don’t trust the numbers!
Subscribe to Bob Watson’s blog, Docs by Design.
Watch my talk or read my blog about adding data to your documentation prioritization process.
Level up your data analysis by exporting your data to a more robust tool where you can do advanced analysis and visualization.
Dive into the resources on a site like Analytics Demystified to get more familiar with the analytics platform that you’re using.
Explore other data sources, such as metadata about your documentation like topic length, header length, readability, and more. Kumar Dhanagopal covers this in his talk as well.

If you’re using Google Analytics 4, get familiar with what you can do in Reports and Explorations. ↩︎
For example, say you’re using Google Analytics 4 and your documentation is all on one domain. You can filter on hostname and focus on only that content. If your content is only relevant for a specific path, well, now you’re struggling to build a filter in GA4 because it looks for exact regex matches by default, whereas UA did partial matches and also supported in-table regex filtering. If you need to match a variety of content such as /docs/guide/admin-intro.html, /docs/guide/configuration-guidance.html, /docs/guide/config-details.html, /docs/guide/administrator-reference.html and others, I recommend using Explorations and creating a custom Events Segment with regex matches for the paths. In this case, one condition would be Page path and screen + matches regex + /docs/guide/admin.* OR Page path and screen + matches regex + docs/guide/config.* to match all the page paths starting with admin or config and map to the relevant topics. Create as many filter conditions as you need, and maybe wish that past you (or another past writer) was a bit more consistent with topic title naming in the URL path! ↩︎
As Kumar Dhanagopal points out in his talk at Write the Docs Portland 2022, Don’t trust the numbers!, Offline and secondhand usage is missing from the data. Anyone who blocks cookies isn’t tracked, and people who download PDFs of the documentation aren’t recorded either. Remember that website analytics data do not provide an exhaustive accounting of people viewing your documentation. ↩︎
The Google Analytics 4 documentation on Default channel grouping provides the rules that comprise these definitions, but the descriptions aren’t that helpful. Thankfully, a blog on How to Change Default Channel Settings in Google Analytics has useful descriptions. ↩︎
Page referrer is a parameter and not a top level dimension that you can add to reports in Google Analytics 4. As a result, you’ll want to set it up as a custom dimension so that you can add it to tables. See the Google Analytics 4 documentation on Custom dimensions and metrics. ↩︎
Google encrypts user searches, and as a result, doesn’t share individual search terms that might lead users to your site with any analytics tools including its own. ↩︎
This blog post on Search Engine Land about How to get the most out of the Google Search Console API using regex provides useful guidance for making sense of Google Search Console. ↩︎
For a definition of total users and active users, see the Google Analytics 4 documentation on Analytics dimensions and metrics ↩︎
Google Analytics 4 includes a scroll event with a percent_scrolled parameter, but knowing how far someone scrolled on your page doesn’t tell you if they were reading your documentation. Because documentation is usually scanned or referred to rather than read, identifying the scroll percentage for a documentation topic is not helpful. ↩︎
Tucker FitzGerald addresses these two types of patterns in depth in an article for UX Booth, Searchers and Browsers: the Personality Types of UX. The excellent entry on How people find information in the Australian Government Style Guide pointed me toward his article. ↩︎

Tips for writing SaaS documentation

Wed, 26 Oct 2022 22:17:57 +0000

If you’re writing documentation for a software-as-a-service (SaaS) product that releases constantly, it’s easy to be overwhelmed by the amount of new functionality being released that needs to be documented.

But do you really need to document all of it? Nope.

Instead, document the hidden, document the weird, and document the why.

Consider every new product feature in terms of what the user cares about. Don’t document what the product can do—document what the user wants to do.

I recommend adopting minimalism in your documentation, especially if you’re trying to keep up with the fast-paced release cycle of a SaaS product. Anni Bond wrote an excellent overview on Adopting minimalism in your docs.

I use a series of questions to keep my documentation minimal and to focus my documentation efforts.

Documenting a feature

When reviewing a feature, consider the following:

What might confuse someone about this?
What could go wrong that they’d need help fixing?
What is different enough from existing product experiences that might require shaping a mental model?
Where do these changes fit in a customer workflow?
In what ways could the feature be improved to be clearer or easier to use (without documentation)?

It’s often helpful to use your role as a user advocate to push for product improvements rather than documenting around problems with the user experience.

Reviewing doc feedback

When reviewing documentation feedback from a customer or the field, consider the following:

Was more than one person confused?
Is this feedback related to one specific edge case?
Is this an actual error? How much time will it take to track down?
What led to someone being confused? Is their personal mental model different?
Is it actually confusion about how the product works and isn’t necessarily a documentation task?

Try to understand the root of the feedback. Rather than giving in and adding a quick note about anything and everything, consider what improvements might make sense to make to the documentation or the product to address the core issue.

Documenting strange behavior

When reviewing a request from PM or engineering to document something weird, consider the following:

Is this time-limited?
- Is this behavior only going to be true until a certain point of time?
Is this feature-limited?
- Is this behavior only going to be true until a feature gets refactored/improved/enhanced?
- Is that work scoped and planned for anytime soon?
Is this actually a bug that should be in the release notes and nowhere else?
How many people are affected by the weird thing?
How common or likely is the weird thing to occur in actual customer environments?

Write less for success

It might seem like I’m telling you to find a lot of reasons to avoid writing documentation. I am. It’s important to focus on the documentation that customers are going to go looking for, instead of making sure the entire product functionality is documented.

Say your product requires people to confirm their email address before they can start using it.

Will customers read documentation about how to confirm their email address? No.

Do you need to write documentation about how to confirm an email address? Also no.

If you have a strong understanding of your users and their mental models, you can make this same determination for other, more complex, workflows in your product.

Stop trying to keep up with the product and start helping the people using it.

Technical documentation as a map

Mon, 19 Sep 2022 20:46:08 +0000

Matt Webb wrote a post about organizing data and mapping the experience of the web, and that made me consider how the decisions about documentation structure, especially in the early stages, require similar decisions.

Technical documentation functions as a map for your product. For specific users, your documentation provides wayfinding guidance and provides the information necessary to navigate the space relevant to your product.

Beyond a map for your product, technical documentation can also map out specific workflows within your product—specific routes, one might say.

For example, diagramming and describing how data flows through the 55+ different systems that comprise Facebook. Without that documentation, you might find yourself in the same situation as some Meta engineers did in March¹, admitting that:

“we have a somewhat strange engineering culture compared to most where we don’t generate a lot of artifacts during the engineering process. Effectively the code is its own design document often.” If the code serves as the documentation, that’s the same as having street signs or individual transit stations, but no sense of where you are within a larger system. You need a map to add a layer of intelligibility to the individual parts of an overall system.

Considering your documentation as a map can help you when you’re struggling to know where to start when writing minimum viable documentation.

What kind of map might you draw on a napkin about your product, or about a specific data interaction or user interaction?

What is the key wayfinding information? What are the intersections or landmarks to be aware of?
What would confuse someone if you left it out of the map? Are your steps complete enough to help someone get to their desired destination?
Do you understand the destination and the person you’re guiding well enough to draw such a map?

If you consider those questions the next time you need to write documentation about a product, a data flow, or a user flow, you and your customers will be much better situated.

As disclosed in a court filing reported by The Intercept in Facebook Engineers: We Have No Idea Where We Keep All Your Personal Data and further reported on by Vice in Facebook Engineers Admit They Don’t Know What They Do With Your Data. ↩︎

Write better docs with a product thinking mindset

Mon, 25 Jul 2022 15:30:00 +0000

I’ve frequently seen product thinking discussed in product management and user experience design contexts, but haven’t seen it applied to technical writing and documentation. And yet, by applying product thinking to documentation, we can write more useful, relevant, high quality documentation.

What is product thinking?

Product management thought leader Shreyas Doshi defines product thinking as follows in a Twitter thread:

“Product Thinking is about understanding motivations, conceiving solutions, simulating their effects, and picking a path based on the effects you want to create.”

As a technical writer, I focus on user problems and the customer journey as a central part of my job, so it seems obvious to adopt product thinking for documentation. While I was working at Splunk, the documentation culture fostered a product thinking mindset, enshrined in the Splunk docs book, The Product is Docs.

Despite being immersed in that culture and caring deeply about helping users, I still found it easy to fall into a “just get it done” mindset, or what Doshi calls project thinking:

“Project Thinking is about understanding expectations, formulating plans, marshaling resources, and coordinating actions to meet those expectations.”

Where project thinking focuses on the steps and process for creating a product, product thinking focuses on the motivation for creating a product and the possible outcomes of that motivation.

Why project thinking is pervasive in technical writing

Technical writers are often short on time and resources. Forced to keep track of multiple teams and projects, we often spend what energy we can on the priorities that seem most urgent. Even with sufficient resources and planning, time crunches still happen. With these limitations, it’s easy to focus solely on churning out documentation and just keeping up.

Sprinting alongside engineers, we often adopt engineering practices like breaking our work down into small parts, writing tickets and adding them to sprints to track with developers. With the widespread adoption of CI/CD pipelines in product development, these practices seep further into our documentation processes.

It feels like an optimization shortcut to consider documentation projects in terms of “feature coverage”, much like how quality assurance projects are considered in terms of “test coverage”. Documentation can become another box to tick, making sure that every pull request deployed has sufficient test coverage and documentation coverage for the product functionality.

This “coverage” mindset is ruled by project thinking. What if we took a product thinking approach?

Project thinking vs Product thinking

While project thinking leads us to write documentation that ensures coverage a specific product feature, product thinking helps us focus on what needs to be written and why. Embracing product thinking lets us write higher quality documentation that is more relevant to customers.

That probably seems too good to be true, so let’s look at a specific example—a product making changes to the API endpoints available to developers as part of a versioning change.

Project thinking for API endpoint deprecation

Taking a project thinking approach to the documentation might look like the following:

documenting each new endpoint
making sure each deprecated endpoint has a link to the relevant new endpoint

There might be also be a deprecation plan in the project that includes “Add a banner to the old endpoints with a link to the new endpoints indicating that they are deprecated and will be removed in the future”.

As a project thinker, this approach has the essentials:

Stage	Action
Expectations	Document the new API
Planning	Docs and engineering work together to produce updated documentation Provide deprecation information to users
Resources	Docs and engineering
Delivered Output	Documentation available with new API release and shared deprecation plan

This approach focuses primarily on coverage for the new functionality, and offers some form of continuity for users using the old functionality by alerting them to the change.

Product thinking for API endpoint deprecation

Taking a product thinking approach to the documentation might instead look like the following:

As soon as you know API endpoints will be changing, write learning outcomes such as the following examples:
- As a developer, after reading the API docs I can successfully update my code to use the new endpoints.
- As a developer, after reading the API docs I am confident that I won’t lose any data and am correctly handling data formats sent by the new endpoints.
- As a developer, after reading the API docs I can explain to my colleague who was on vacation which endpoints changed.
- As a developer, I want to explain to my boss when I need to deploy my new script (when the API changes become permanent).
Documenting each new endpoint.
Making sure each deprecated endpoint has a link to the migration guide.
Writing a guide for users of the v1 API to help them migrate to the v2 API, that addresses the specific learning outcomes you defined by including the following:
- A note or caution that calls out any changes to data formats.
- Specific code examples comparing API calls using the deprecated endpoints with the API calls using the equivalent new endpoints and parameters.
- Clear dates or communication methods where developers can find out when the v1 API stops working.

From a product thinking perspective, this list of steps covers the essentials:

Stage	Action
Motivations	Identify the customer motivations by writing learning outcomes
Solutions	Make a list of documentation options that address customer motivations.
Simulations	Write a migration guide and draft specific code examples to validate with engineering and product management.
Desired Outcomes	Customer achieves business continuity and engineering can deprecate old API endpoints and stop maintaining the code.

This approach focuses primarily on customer outcomes, rather than optimizing for producing a specific type of documentation or specific output that can be checked off a plan.

Comparing the results

If project thinking optimizes for coverage and output, product thinking optimizes for outcomes. The documentation written from a product thinking mindset seems like it’s a lot more detailed, but also slow and resource intensive to create. And it is.

But product thinking is only slow in the short term. If you focus on coverage (and output) instead of outcomes (and customer success), you might have complete documentation, but it’s not as useful to customers. Over time, you might not have many of those to worry about.

I’m not advocating for abandoning project thinking entirely. We can’t neglect resource considerations or the content delivery timing of what we’re writing, especially if we’re working as lone writers. But just as it’s crucial to consider the timing of what we write, we need to consider the usefulness of what we write. Cultivating a product thinking mindset is essential to writing useful documentation.

Applying product thinking to documentation

It’s easy to fall into the habit of being a “reactive” documentarian, responding to requests for information and rotely updating page after page after page with more information. As technical writers, our priority lists are often shifting or shuffling around. So how do you go about applying product thinking to your documentation?

Approach the documentation itself like a product.

Create cues in your writing process, get access to people with essential information, write learning outcomes for your documentation, consider the content design of your documentation, and test everything about your proposed changes!

Create process cues

To make sure you don’t spend too much time on “getting the writing done”, add cues to your documentation process that can help you maintain user-centricity in your writing. What kind of cues? It depends on how you do your writing, but here are some that have worked for me:

Follow a template for documentation tickets. On one fast-paced development project, I made sure that every documentation ticket I filed included the following:

the audience of the content
the technical SMEs that knew more about the feature
the learning outcome(s) for the content

If I couldn’t fill out those details, I knew I couldn’t work on the documentation request without producing poor quality content.

Review and plan your work regularly. On another project, I embraced the essentials of project thinking by spending some time on Friday afternoons reviewing what I’d accomplished in the last week and planning out my tasks for the following week. An output-focused approach that I combined with an outcomes-oriented approach by identifying the key product outcomes or customer goals associated with each documentation task I planned to work on.

Get access to people

It’s impossible to write product-focused documentation with customer outcomes in mind if you don’t have the access to the product managers, engineers, and customers to understand and validate those outcomes.

Make sure you can talk to the product managers, engineers, and UX designers working on your product, but especially take steps to build relationships with sales and customer support representatives that interact directly with customers every day.

With access to product experts and customer experts, you can more readily identify and validate the customer motivations driving the development of specific product functionality. This is especially crucial in an agile development environment, where the functionality that you’re documenting might only be a small building block of a larger solution, but in isolation it seems clunky and broken. For example, an incident management feature that adds the ability to store information about an incident, but neglects a status field to denote the state of the incident.

With a deep understanding of the why and why now behind product development decisions, you can structure and write your content in a way that accommodates customer goals alongside these possible product limitations. For example, you can write an incident management workflow that includes options for adding a status to an incident. In this way, you’re writing documentation that addresses the existing feature, the customer’s desired outcome, and leaves room to showcase future product functionality.

Write learning outcomes

It’s crucial to understand the customer’s desired outcome when using a product. As the technical writer for a product, however, you also want to establish the customer’s outcomes when using your documentation.

When you write a learning outcome, you want to use active words and be specific. These aren’t abstract user stories, but rather concrete outcomes that customers can achieve.

Don’t write this:

After reading the docs, I understand how to call the API.
After reading the docs, I understand how to create an incident in the product.

Instead, be specific about the desired customer outcomes:

After reading the docs, I can successfully update my code to call the new API endpoint
After reading the docs, I can confidently create an incident based on a phone call from DevOps.

You need to write specific learning outcomes for your documentation so that your documentation can also be specific and useful to customers.

If you can’t write a compelling and specific learning outcome for a feature, several things could be true:

You might not need to document it. If there aren’t any associated customer outcomes, it probably doesn’t need documentation.
You might not understand the customer motivations well enough. Check in with product management and the technical SMEs for the feature to dig into why the feature is being added to the product to uncover the motivations driving development.

Writing learning outcomes for every piece of documentation you produce (including the small ones) helps shift you to a product thinking mindset and reminds you why your documentation is important.

Consider content design

Another element of the product thinking mindset is considering new ideas. Content creation can feel routine and straightforward, but it involves design just as much as it involves writing.

Considering the content design of your writing can make your writing more accessible, more inclusive, and more useful. If you include more headers or change paragraphs to a table or a list, your content is easier to scan and therefore more helpful to anyone easily bored, in a hurry, and with dyslexia and/or ADD.

You can also break up paragraphs with visuals such as diagrams or screenshots, with helpful alt text, which can reinforce your content and provide extra learning opportunities for more visual learners.

In addition to adjusting the design of your text, you can also change how and where you present your content. Consider the following alternatives to text documentation:

An interactive product tour
A blog post
A 2-3 minute video
An interactive tutorial that uses a demo environment

For example, you could write a blog post to walk through incident management functionality in your product using a common IT incident or a security exploit from the news. This alternate form of content provides a tangible relevant example to customers in a dated blog post that you won’t have to keep updated.

Test everything

To continue treating documentation like a product, you need to test out the decisions that you make along the way. Validate the product motivations you’ve identified, the customer learning outcomes you’ve defined, and take advantage of any and all testing options available to you.

Get a review from technical SMEs about the accuracy of your content.
Ask a peer for feedback on the presentation of your content, especially if you’re trying something new.
Send your writing to get edited, if you have the luxury of access to an editor.
Check your writing for style guide adherence and completion, using a tool like Vale or similar. I’m guilty of publishing docs with broken links and unfinished sentences. Automated or manual checklists are crucial to make sure your work is finished.
Perform UX testing on the content. If you restructured the information architecture, use tree testing to test out the intuitiveness of your changes.

Write better docs with a product thinking mindset

With a product thinking mindset, you build a clear understanding of why you are documenting something. In the end, that clear understanding produces documentation that provides a valuable resource to customers and can even prove to be a differentiator from your competitors.

From Nothing to Something with Minimum Viable Documentation

Tue, 21 Sep 2021 15:50:00 +0000

More and more startups and enterprises are recognizing the importance of high quality product documentation, but it’s tough to know where to start. I’ve taken a few enterprise software products from “nothing to something” documentation and this is the framework I’ve built for myself to create MVD—minimum viable documentation.

If you’re a technical writer trying to find your footing, or someone who cares about adding user documentation for your software but have no idea where to start, this is the guide for you.

What is minimum viable documentation?

If documentation is a product (and it is), minimum viable documentation is the bare minimum documentation that is useful and helpful to customers.

Something good is better than something chaotic and unhelpful, and it’s much better than no documentation at all. It’s also easier to focus on getting to minimum viable documentation rather than trying to reach full-featured documentation as soon as possible, because you’re a human with a life that is not your job.

You might be working with a fully-functional software product that has no useful documentation. In that case, getting to full-featured documentation isn’t your primary goal—getting to minimum useful documentation is. So let’s get started.

Define minimum viable documentation for your product

Before you can write MVD, you need to define what it is for your product. MVD differs depending on your market, customer base, product type, your pricing structure, and more.

I recommend you do the following to define what MVD looks like for your product.

1. Talk to your colleagues

Your goal with these conversations is to get a good understanding of who the target user is for your product and the goals they want to accomplish with your product.

If you have product management, start with them. Find out as much as you can about why the product is being built, who it’s for, and how the product is being positioned in the market.

Also talk to engineering management or senior engineering subject matter experts (SMEs). What user problems are the software trying to solve? What level of expertise do the engineers assume the user has?

If you’re lucky enough to have a sales or marketing team, talk to them. Because of their efforts defining the customer journey, they can help you understand who the audience is and what the key success workflows look like. Who is the product targeting? Why do they want to use this product? What problems are they trying to solve?

Talk to the user experience designers to get an understanding of the user personas they’re designing for and what workflows they think have the most friction. You can also get a sense for how the team approaches their role, whether they’re more focused on designing friction-free workflows or pixel-perfect screens.

After talking to PM, EM, UX, and marketing, you can do the following:

Identify what level of expertise a typical product user has, both with the domain and with the product. This functions as your audience definition.
Write down the main goals of a user before and after they start using your product. What motivates the user?
Map out the key workflows that a user is going to perform in the product. What tasks are the user trying to accomplish?

2. Perform a documentation competitor review

It’s always a good idea to know what your competitors are doing! If you’re not sure what products your product is competing with, ask your sales or marketing people for a list or do some research on your own.

Pick 3-5 companies to focus on, such as your strongest competitors in terms of funding, usage, or closeness to what your product does.

You also want to make sure you’re not benchmarking off useless garbage when you perform your competitor review. In addition to the 3-5 competitors you identify, pick a couple industry leaders or companies that your colleagues mention as having good documentation, such as Stripe API docs, Microsoft docs, or even IBM docs, and include one or two of them in your competitor analysis.

The advantage of choosing the documentation of a couple larger products to review is that they tend to have established documentation teams and offerings in a variety of markets. This makes it easier to find a product that is well-documented and at least somewhat adjacent to what your product does.

The goal of your competitor analysis is to identify how the companies provide documentation about their product(s). Pay attention to the following:

How is the documentation structured?
- By feature?
- By use case?
- By persona?
What documentation is provided?
- What workflows are covered?
- What seems left out?
What type of documentation is it?
- Lots of conceptual information about how the product works?
- Heavy reference information but light on the how-to tasks?
Who is the documentation targeting?
- Look for introductory content or tutorials
- Is there advanced developer content?
How is the documentation site built?
- Use the inspect element option in your web browser or a site like https://builtwith.com/ to figure out what technology the documentation site is built with.
Anything else interesting?
- Does a company have an interesting way of differentiating beta functionality?
- Are code samples hidden behind toggle-to-expand options?
- Is there a plethora of gifs, videos, or other multimedia in the documentation?

Document your findings, of course, and feel free to share your findings with your team during a demo. After all, they might be wondering what you’re doing if you haven’t started writing yet.

3. Assess the current state of your documentation

If there is any sort of documentation for your product, you want to know what it is. It might be a sad README and some code comments, it might be detailed multilayered documentation without much organization or clear goals.

To get a sense of the current state, I recommend doing the following:

Audit the existing content. Identify which topics are covered in the documentation already, and where. Make a list, and also keep track of what topics seem to have a lot of detail, and what you suspect might be outdated. This is a cursory audit, not an in-depth one that you might perform if you were migrating content.
Look at the documentation analytics. If you have analytics for the documentation site, take note of which pages are most frequently viewed, which pages might be serving as entry points, and how much time people spend on various pages.
Talk to your team and get their thoughts on the current documentation. Who has been writing it so far? Are they attached to any topics in particular? Do they share specific topics with customers regularly?
Interview customers of the product and documentation to see what they want to see or find most useful today.

Depending on the quality of the existing documentation, these steps might not be that helpful in informing your approach, but help you set benchmarks for documentation growth and quality, plus identify links you likely don’t want to break.

If you don’t have any documentation, still talk to your team and customers. If you can’t talk to customers for some reason, you can look for discussions about the product on social media like Reddit, Twitter, or Hacker News to identify themes that people ask questions about or really enjoy about your product.

A brief note about terminology: As you review competitor and existing documentation and interview internal and external folks, you might find that your product has some inconsistent terminology. At this stage, you might want to delay the writing process while you create a definitive list of terms to use for the product. This type of work can take more time upfront but it’s easier to create consistency from the beginning than to apply it after the fact.

Define the structure of your documentation

Before you start writing, you want to create a structure or a framework to place your topics into.

The structure for your MVD is directly informed by the work you did to define what MVD looks like for your product, plus some information-architecture-specific research.

Revisit your conversations with colleagues. What workflows and functionality might be important to highlight? Who is buying your product? Who is using your product?
Refer to your competitor review notes. How did your competitors and benchmark docs structure their documentation?
Research information architecture best practices. Refer to some key articles from the Nielsen Norman Group, as well as the book How to Make Sense of Any Mess by Abby Covert, and the associated worksheets.

After this research, draft up some chapter headings and possible topic titles to start with, then get feedback from your UX, PM, Engineering, and Sales and Marketing folks. How accurate, relevant, or helpful does the new structure seem? Have you made any assumptions that don’t make sense for the customer base?

Expect this information architecture to change as you write the MVD and especially as you develop full-featured documentation. This is the nature of a minimum viable product! Put a task in your backlog to plan to refine the structure after you finish MVD and are approaching full-featured documentation so that you can iterate without confusing your customers with frequent changes, and plan so that you don’t break any links.

After you design the initial information architecture, you can start writing.

Start writing minimum viable documentation

So you know what minimum viable documentation might look for for your product, but how do you get there? MVD is all about creating useful content for your users, so start with the entry content!

Focus on key information for customers

As with any “minimum viable” approach, you’re trying to get a basic functional framework down before you start improving it. As you lay that framework, be mindful of scope creep.

Think back to the key workflows that you mapped out earlier. Broadly cover the top few workflows and then flesh out details as you get more comfortable with the product and understand the user goals better. Why go broad instead of going deep into a specific workflow? You’re still learning what the customer finds useful, and what level of detail they might want or need about a specific workflow.

If you spend a lot of time writing a highly detailed workflow that you thought was important and it turns out it’s actually pretty intuitive for customers—that’s time that you could have used to write about something that was really confusing and holding back customers from succeeding with your product.

It’s likely that you’ll encounter cases and situations that you want to write more about. That’s great! Write them down and put them in a backlog to address later. For now, you want to stay focused on these minimal workflows to build out the minimum viable documentation for your product. You can get fancy with use cases and in-depth examples later.

Identify the simplest path to success

Within those broad key workflows, start with the simplest path to success, the “happy path” that most of your customers will take.

That might involve writing a series of topics like:

“Get started using Best Product Ever”
“Install Best Product Ever”
“Set up Best Product Ever”
“Accomplish Straightforward Task in Best Product Ever”.

Get those written, reviewed, and published and start helping people use your product that much sooner.

Clarify any complexity

After you write the documentation to support the simple path to success, what do you write next? Documentation that unravels where complexity lurks in your product.

Depending on your product familiarity, you might need to take more time to research and lean on technical subject matter experts (SMEs) a bit more to write this, but it’s worth it. This documentation content might be topics like:

“Configure the Weird Setting You Must Touch”
“All About This Task That Everyone Wants to Do but No One Can Find”.

You don’t want to get bogged down in documenting around product complexity here. Stay focused on the complex aspects of the key customer workflows, and the crucial information customers need. What might confuse someone if you left it out? What assumptions have you been making about the user that need to be made explicit?

This is often the step when I remember to write things like software requirements, role-based restrictions to functionality, or other crucial cases that are often assumed when developers write their own documentation.

Get feedback and iterate

I assume you’re focusing on minimum viable documentation because you have more work than you have time to complete it. That’s why it’s important to iterate. Yes, I just harped on the importance of prioritization and focus—and it’s essential to make sure that what you prioritize and focus on is still important.

Check in with product management and engineering management regularly (I’d recommend weekly for an every-few-months or less release cadence) about what you’re prioritizing and why.

This check-in is mostly about getting signoff and validation, not direction—but don’t ignore the direction that PM and EM can offer you! If there are important releases coming up that will affect one of the key workflows on your list, you might want to document that workflow sooner, or in more detail than you might otherwise for MVD.

Use these conversations as a way of discovering what customers are paying attention to, and what your PM and engineers are paying attention to as well.

As you send your documentation out for technical review, you might also get feedback that you can use to improve your approach to MVD. With any luck, much of the feedback will duplicate what you have planned—and that’s helpful validation for your approach.

You might get so much feedback that you have to dump a lot of ideas into “plans to write this later” and a backlog that feels like it’s spiralling out of control, but if you stay focused on your scope, you’ll get to that backlog sooner and with a more comprehensive understanding of your documentation and your customers.

If the direction and feedback you get from your team is pretty far removed from your approach to MVD, it’s helpful to discuss why that might be and treat it as prioritization guidance for your future plans. Maybe you misunderstood a key target customer, or the purpose of the product in the market. You might discover you need to realign your understanding and vision of the documentation with that of your team.

What’s next after MVD?

When do you know that you’ve reached minimum viable documentation? It’s somewhat of a fuzzy line. When you notice that you’re writing documentation by adding to existing topics, or writing net new example content, or documenting new features instead of existing features — you’ve moved past MVD and into shaping full-featured documentation.

As you start shifting into that mode, you’re no longer focused on creating the skeleton structure to build off of, but filling in the details and settling into the usual work of modern technical writing.

1. Go through the backlog

Start going through your backlog of ideas. Revisit those ideas and group similar ones together, adding audience definitions, acceptance criteria, and learning objectives where you can. Note who the technical SMEs are and whether any upcoming releases are relevant for some of the tasks.

Ideally, you’re storing this backlog in the same spot as your engineering backlog so that your work is visible to the engineering team.

Work with PM or EM to prioritize those tasks and start working through them. As any writer for a fast-paced development team knows as well, the product development often happens faster than you can write about it, so you’ll never run out of tasks in your backlog.

2. Suggest product improvements

As you went through a flurry of documentation writing to produce MVD, you likely identified some parts of the product that might need to be improved. Again, work with your PM and engineering teams to discuss possible product improvements.

You can also suggest product improvements that directly involve the docs, doing a review of UI text in the product, or auditing pages in the product to suggest opportunities for in-app documentation or context-sensitive help. This is a great opportunity to partner with the UX team as well.

Partner with your engineering and UX teams to make suggestions and build those relationships based on your newfound product and customer expertise.

3. Write use cases and examples

To create more useful content for your customers, you probably want to flesh out specific example scenarios for using your product. You might have written some already as quick start use cases for getting started with your product, but you likely want to write more for the next stage of customer product understanding.

You can use example content to describe customization options for the product, or highlight domain-specific use cases for a market that your customer might be trying to break into.

4. Ask for feedback

You put all this effort into creating minimum viable documentation, but how viable is it really?

Ask your technical SMEs, sales and marketing teams, customers, really anyone that might interact with the documentation internally or externally if they have feedback on your documentation improvements and information architecture.

You could perform some tree testing with the MVD structure to see if there are some improvements you can make to the information architecture as you flesh out the documentation, or just have short conversations with stakeholders.

Use the feedback you get to help shape priorities for your backlog. However, don’t treat all feedback you get as tasks that you must perform—if someone asked for it, it must be important, right?

Instead, validate feedback against your target audience definitions and user goals. Sometimes you’ll get feedback relevant only to a specific edge case that doesn’t make sense to document in the official documentation, or feedback related to a product bug that isn’t something necessarily appropriate to address in the documentation.

5. Review analytics

Review documentation site analytics. Analytics are an imperfect source of feedback, but as long as you established a prior benchmark, check to see if the entry-level pages that you created or updated are the most popular pages.

Are the pageviews higher, or at least somewhat proportional to the user base of your product?
Are there any surprising outlier pages that have a lot of views that you might want to focus on?
What search terms are popular?

You can use these to inform your plans and priorities.

Get from nothing to something with MVD

It can be intimidating to create a set of documentation for a product from scratch, but I hope this post outlines a basic approach that can help.

Start by defining what MVD looks like for your product by talking to colleagues, performing a competitor review, and assessing the current state of documentation. Then do some additional research and define the initial structure of your documentation.

After you’ve laid the groundwork, start writing. Focus on key information for customers and identify the simplest path to success. Clarify any product and task complexity, and seek out feedback. Regularly make changes to what you’ve written as you learn more about the product and your customers.

As you evolve beyond MVD to full-featured documentation, work through your backlog, suggest product improvements, write use cases and examples, and continue asking for feedback. You can also review site analytics to get a sense of how far you’ve come and what you might want to focus on next.

Whether you’re a professional technical writer, a committed startup founder, a generous open source contributor, or someone else, I hope you can use this framework to document your software product.

I tried my best to create a minimum viable blog post to describe this minimum viable documentation framework. As such, I might not have gone into much depth about how to perform a competitor review, get buy-in for terminology proposals, or how to handle the full range of feedback you might receive on your documentation.

If you have feedback or questions for me, or want to see more details about a specific topic, don’t hesitate to reach out on Twitter @smorewithface.

How can I get better at writing?

Wed, 23 Dec 2020 07:41:57 +0000

As a professional writer, I frequently get asked, “as a ______, how can I get better at writing?” I’ve never had a good list of resources to point people to, so I finally decided to write one. I’ve worked hard to become a good writer, and I’ve had the privilege of many good teachers along the way.

If you’re not really sure why your writing isn’t as good as you want it to be, that’s okay. In this blog post, I’ve identified the strategies that I use to write well. I hope they’re useful to you.

Where to start

Read and write more frequently. You can’t get better without good examples or practice. If you want to get better at writing you need to read more and you need to write more.

Identify what you’re trying to improve. Maybe you struggle with grammar, or in clearly communicating your ideas. Maybe it takes too many words for you to get your point across, or you can’t quite connect with the people reading your writing.

Write accurate content by improving your grammar and word choice

Use a tool like Grammarly, or enable grammar checking in whatever tool you use to write, if it’s available. If you don’t want a mysterious AI reading your writing, you can use other resources to improve specific aspects of your grammar.

Some key concepts to focus on:

Understand verb and noun preposition pairs. See Kenneth Beare’s guide Verb + Preposition Combinations on ThoughtCo., or this Preposition Combinations with Adjectives, Nouns, and Verbs PDF from Washtenaw Community College, including a worksheet for practice.
Understanding complementary verb and noun forms, to help avoid nominalizations. When you use the noun form of a word, you can easily obscure your meaning. See How to Improve Your Writing: Avoid Nominalizations on Wordvice.
Review easily-confused words, often homonyms, to identify words that you might be consistently misusing. This Homonyms Main List from an English learning Russian website is fairly exhaustive.
Look words up in the dictionary. If you’re not sure you’ve used a word correctly, or used the correct spelling of a word, look it up. I do this almost every day. I’m partial to the Merriam-Webster dictionary.

I still struggle with the following (more pedantic) grammar rules:

When do I need to use a hyphen to connect two words? See Hyphen Use, on the Purdue Online Writing Lab website.
Did I split an infinitive? What is a split infinitive, anyway? See Infinitives, on the Purdue Online Writing Lab website.
Does my relative pronoun actually clearly refer to something or do I have a vague “that” or “it”? See Pronouns in the Splunk Style Guide.

The somewhat silly yet practical book, Curious Case of the Misplaced Modifier by Bonnie Trenga, might also be a useful read.

Write helpful content by defining outcomes before you start

Before you start writing something, whether it’s a slide deck, an engineering-requirements document, an email, or a blog post like this one, consider what you want someone to do after reading what you wrote.

Often called learning objectives or learning outcomes in instructional design, defining outcomes can help you write something useful and focused. Sometimes when you’re writing something, other extraneous ideas come to mind. They can be valuable ideas, but if they distract from your defined outcomes, you might want to remove them from your main content.

Some example outcomes are:

After reading this blog post, you can confidently draft a clear document with defined outcomes.
After reading this engineering requirements document, my colleague can provide accurate and helpful architecture feedback on the design.
After reading the release notes, I can convince my boss that the new features are worth an immediate upgrade.

I also want to note that if you write an outcome focused on someone understanding something, rewrite it. It’s tough to measure understanding. It’s easier to measure action. For that reason, I try to write outcomes with action-oriented verbs. For more about writing good learning objectives, see the Learning Objectives chapter in The Product is Docs.

Write focused content by identifying your audience

Who will be reading your writing? What do they know? Who are they? What assumptions can you make about them?

If you can’t answer these questions about the people reading your writing, you won’t be able to clearly communicate your ideas to them. You don’t have to be able to answer these questions with 100% certainty, but make the attempt.

If you recognize that you’re writing something for multiple audiences, consider breaking up the content into specific sections for each audience. For example, architects might care about different content than a UI engineer, a product manager might care about different details than the backend engineer.

If you identify the different needs of your varying audiences, you can write more consistently for each specific audience, rather than trying to address all of them all the time. For more on identifying your audience, see the Audience chapter of The Product is Docs.

Write findable content by considering how people get to it

How people get to your content can influence how you write it. If people use search, an intranet, or direct links to find your content, you might make different decisions about how to structure it.

I always assume that people are finding my content by searching the web. They’ve typed a specific search query, found my content as a result, and open it with the hopes that it is the right content for them.

Consider what people are searching for that can be answered by your content, and write a title accordingly. Spend time on the first few sentences of your content to make sure that they further clarify what your content addresses.

For example, I titled this blog post “How can I get better at writing?” because I expect that’s what a lot of people might type into their preferred search engine out of desperation. I could call it “7 quick tips to improve your writing”, but that’s not how most people type search queries (in my opinion).

Mark Baker’s book, Every Page is Page One, covers a lot of information related to this concept. He coins the term “information scent” to describe the signals that indicate to a person that they’ve found the right content to answer their question, and “information foraging” to describe the process of looking for the right information.

Write readable content by considering the structure

People aren’t excited to read technical content or technical documentation. No one rejoices when they get an email. I get paid to write technical documentation and I still avoid reading it if I can. Because people don’t want to read your content, structure it intentionally.

Write for skimming. Bullet points are often better than paragraphs. Tables are often better than paragraphs.

Put information where it needs to be. If you’re writing a series of steps, make sure the steps are actually in the right order. For example, if something needs to be done before all the steps can succeed, put it before the set of steps as a prerequisite.

You also want to consider the desired outcomes of your content and your audience when you structure your content. It can make sense to focus on one audience in one piece of content, or one desired outcome in one piece of content. Don’t try to do too much in one piece of writing.

Nielsen Norman Group has an incredible set of research and recommendations about how people read and how you can structure your content. I recommend the following articles:

Write clear content by intentionally choosing your words

You want to make your content easy to find and easy to understand. To do this, you need to be consistent and intentional about the words that you use.

Use consistent terminology. This isn’t the time to write beautiful prose that uses different words to mean the same thing. Don’t overload terms by using the same term for multiple things, and don’t use multiple terms to refer to one thing. Use the same terms and use them consistently.

If something is a JSON object, call it that. Don’t call it a JSON object sometimes, a JSON setting other times, or a JSON blob other times. Pick one term and use it consistently. You might have to pick an imperfect term and live with it. It happens! There are only so many words to choose from.

Be intentional about the words you use. Consider the words that your readers use to describe what you’re writing about, and use the same words if you can. Even if those words don’t match up completely with the feature names in use by your product.

If all of your software’s users refer to “dark mode” instead of “dark theme”, you might need to use both terms in your content so that people can find it. For some internal documentation, you might need to make a mapping of internal names that people use for something with the external names used in the product.

If you’re not sure what term to use, find out what terms your readers are already using. If you have access to search query logs of your website search, review those for patterns. If you don’t already have readers or users for your product, you can do some competitive analysis to understand what terms are in common usage in the market.

You can also check the dictionary or use a tool like Google Books Ngram Viewer or Google Trends to identify common terms for what you’re attempting to describe.

Nielsen Norman Group again has some excellent resources on clear writing:

Write trustworthy content by thinking about the future

Errors in content, especially technical documentation, lead to mistrust. When you write a piece of content, consider the future of the content.

The future of the content depends on the purpose and type of content that you’re writing. This list contains some common expectations that readers might have about various content types:

A blog post has a date stamp and isn’t kept continually updated.
Technical documentation always matches the product version that it references.
Architecture documents reflect the current state of the microservice architecture.
An email gets the point across and can’t be edited after you send it.

You must consider the future and maintenance of any content that you write if your readers expect it to be kept up-to-date. To figure out how difficult maintaining your content will be, you can ask yourself these questions:

How frequently does the thing I’m writing about change?
How reliable does my content need to be?
How quickly does my content need to be accurate (e.g., after a product release)?

By answering these questions, you can then make decisions about how you write your content.

What level of detail will you include in your content?
Will you focus your efforts on accuracy, speed, or content coverage?
Do you want to include high-fidelity screenshots, gifs, or complex diagrams?
Do you want to automate any part of your content creation?
Who will review your content? How quickly and thoroughly will they review it?

For more on maintaining content and making decisions about your documentation, see the Documentation Decisions chapter in the book The Product is Docs (which I contributed to).

Feel empowered to write better content

I hope that after reading this blog post you feel empowered to write more accurate, helpful, focused, findable, readable, clear, trustworthy content. This is an overview of strategies. If you want to dig deeper into a specific way to improve your writing, check out the books and articles linked throughout this post.

If you have something you think I missed, you can find me on Twitter @smorewithface.

Detailed data types you can use for documentation prioritization

Tue, 21 May 2019 17:16:22 +0000

Data analysis is a valuable way to learn more about what documentation tasks to prioritize above others. My post (and talk), Just Add Data, presented at Write the Docs Portland in 2019, talk about this broadly. In this post I want to cover in detail a number of different data types that can lead to valuable insights for prioritization.

This list of data types is long, but I promise each one contains value for a technical writer. These types of data might come from your own collection, a user research organization, the business development department, marketing organization, or product management organization:

User research reports
Support cases
Forum threads and questions
Product usage metrics
Search strings
Tags on bugs or issues
Education/training course content and questions
Customer satisfaction survey

User research reports

User research reports can contain a lot of valuable data that you can use for documentation.

Types of customers being interviewed
Customer use cases and problems
Types of studies being performed

This can give you insight into both what the company finds valuable to study (so some insight into internal priorities) but also direct customer feedback about things that are confusing or the ways that they use the product. The types of customers that are interviewed can provide valuable audience or persona-targeting information, allowing you to better calibrate the information in your documentation. See How to use data in user research when you have no web analytics on the Gov.UK site for more details about what you can do with user research data.

Support cases

Support cases can help you better understand customer problems. Specific metrics include:

Number of cases
Frequency of cases
Categories of questions
Customer environments and licenses

With these you can compile metrics about specific customer problems, the frequency of problems, and the types of customers and customer environments that are encountering specific problems, allowing you to better understand target customers, or customers that might be using your documentation more than others. Support cases are also rich data for common customer problems, providing a good way to gather new use cases and subjects for topics.

Forum threads and questions

These can be internal forums (like Splunk Answers for Splunk) or external ones, like Reddit or StackOverflow.

Common questions
Common categories
Frequently unanswered questions
Post titles

If you’re trying to understand what people are struggling with, or get a better sense of how people are using specific functionality, forum threads can help you understand. The types of questions that people ask and how they phrase them can also help make it clear what kinds of configuration combinations might make specific functions harder for customers. Based on the question types and frequencies that you see, you might be able to fine-tune existing documentation to make it more user-centric and easily findable, or supplement content with additional specific examples.

Product usage metrics

Some examples of product usage metrics are as follows:

Time in product
Intra-product clicks
Types of data ingested
Types of content created
Amount of content created

Even if you don’t have specific usage data introspecting the product, you can gather metrics about how people are interacting with the purchase and activation process, and extrapolate accordingly.

Number of downloads and installs
License activations and types
Daily and monthly active users

You can use this type of data to better understand how people are spending their time in your product, and what features or functionality they’re using. Even if a customer has purchased or installed the product, it’s even more valuable to find out if they’re actually using it, and if so, how.

If your product is only in beta, and you want more data to help you prioritize an overall documentation backlog, such as topics that are tied to a specific release, you can use some product usage data to understand where people are spending more of their time, and draw conclusions about what to prioritize based on that.

Maybe the under-utilized features could use more documentation, or more targeted documentation. Maybe the features themselves need work. Be careful not to draw overly-simplistic conclusions about the data that you see from product usage metrics. Keep context in mind at all times.

Search strings

You can gather search strings from HTTP referer data from web searches performed on external search sites such as Google or DuckDuckGo, or from internal search services. It’s pretty unlikely that you’ll be able to gather search strings from external sites given the widespread implementation of HTTPS, but internal search services can be vital and valuable data sources for this.

Look at specific search strings to find out what people are looking for, and what people are searching that’s landing them on specific documentation pages. Maybe they’re searching for something and landing on the wrong page, and you can update your topic titles to help.

JIRA or issue data

You can use metrics from your issue tracking services to better understand product quality, as well as customer confusion.

Number of issues/bugs
Categories/tags/components of issues/bugs
Frequency of different types of issues being created/closed

Issue tags or bug components can help you identify categories of the product where there are lots of problems or perhaps customer confusion. This is especially useful data if you’re an open source product and want to get a good understanding of where there are issues that might need more decision support or guidance in the documentation.

Training courses

If you have an education department, or produce training courses about your product, these are quite useful to gather data from. Some examples of data you might find useful:

Questions asked by customers
Questions asked by course developers
Use cases covered by content in courses
Enrollment in courses
Categories of courses offered

Also useful to correlate this with other data to help identify verticals of customers interested in different topics. Because education and training courses cover more hands-on material, it can be an excellent source of use case examples, as well as occasions where decision support and guidance is needed.

Customer surveys

Customer surveys especially cover surveys like satisfaction surveys and sentiment analysis surveys. By reviewing the qualitative statements and types of questions asked in the surveys, you can gain valuable insights and information like:

What do people think about the product?
What do people want more help with?
How do people think about the product?
How do people feel about the product?
What does the company want to know from customers?
What are the company priorities?

This can also help you think about how the documentation you write has a real effect on peoples’ interactions with the product, and can shift sentiment in one way or another.

Documentation feedback

Direct feedback on your documentation is a vital source of data if you can get it.

Qualitative comments about the documentation
Usefulness votes (yes/no)
Ratings

Even if you don’t have a direct feedback mechanism on your website, you can collect documentation feedback from internal and external customers by paying attention in conversations with people and even asking them directly if they have any documentation feedback. Qualitative comments and direct feedback can be vital for making improvements to specific areas.

Site metrics

If your documentation is on a website, you can use web access logs to gather important site metrics, such as the following:

Page views
Session data like time on page
Referer data
Link clicks
Button clicks
Bounce rate
Client IP

Site metrics like page views, session data, referer data, and link clicks can help you understand where people are coming to your docs from, how long they are staying on the page, how many readers there are, and where they’re going after they get to a topic. You can also use this data to understand better how people interact with your documentation. Are readers using a version switcher on your page? Are they expanding or collapsing information sections on the page to learn more? Maybe readers are using a table of contents to skip to specific parts of specific topics.

You can split this data by IP address to understand groups of topics that specific users are clustering around, to better understand how people use the documentation.

Text analysis metrics

Data about the actual text on your documentation site is also useful to help understand the complexity of the documentation on your site.

Flesch-Kincaid readability score
Inclusivity level
Length of sentences and headers
Style linter

You can assess the readability or usability of the documentation, or even the grade level score for the content to understand how consistent your documentation is. Identify the length of sentences and headers to see if they match best practices in the industry for writing on the web. You can even scan content against a style linter to identify inconsistencies of documentation topics against a style guide.

Download metrics

If you don’t have site metrics for your documentation site, because the documentation is published only via PDF or another medium, you can still use metrics from that.

Download numbers
Download dates and times
Download categories and types

You can use these metrics to gather interest about what people want to be reading offline, or how frequently people are accessing your documentation. You can also correlate this data with product usage data and release cycles to determine how frequently people access the documentation compared with release dates, and the number of people accessing the documentation compared with the number of people using a product or service.

Topic type metrics

If you use strict topic typing at your documentation organization, you can use topic type metrics as an additional metadata layer for documentation data analysis. Even if you don’t, you can manually categorize organize your documentation by type to gather this data.

What are the topic types?
How many topic types are there?
How many topics are there of each type?

Understanding topic types can help you understand how reader interaction patterns can vary for your documentation by type, or whether your developer documentation has predominantly different types of documentation compared with your user documentation, and better understand what types of documentation are written for which audiences.

Topic metadata

Metadata about documentation topics is also incredibly valuable as a correlation data source. You can correlate topic metadata like the following information:

What are the titles?
Average length of a topic?
Last updated and creation dates
Versions that different topics apply for

You can correlate it with site metrics, to see if longer topics are viewed less-frequently than shorter topics, or identify outliers in those data points. You can also manually analyze the topic titles to identify if there are patterns (good or bad) that exist.

Contribution data

If you have information about who is writing documentation, and when, you can use these types of data:

Last updated dates
Authors/contributors
Amount of information added or removed

Contribution data can tell you how frequently specific topics were updated to add new information, and by whom, and how much information was added or removed. You can identify frequency patterns, clusters over time, as well as consistent contributors.

It’s useful to split this data by other features, or correlate it with other metrics, especially site metrics. You can then identify things like:

Last updated dates by topic
Last updated dates by product
Last updated dates over time

to see if there are correlations between updates and page views. Perhaps more frequently updated content is viewed more often.

Social media referers
Link clicks from social media sites

If you publicize your documentation using social media, you can track the interest in the documentation from those sites. If you’re curious about social media referers leading people to your documentation, and see whether or not people are getting to your documentation in that way. Maybe your support team is responding to people on twitter with links to your documentation, and you want to better understand how frequently that happens and how frequently people click through those links to the documentation…

You can also identify whether or not, and how, people are sharing your documentation on social media by using data crawled or retrieved from those sites’ APIs, and looking for instances of links to your documentation. This can help you get a better sense of how people are using your documentation, how they’re talking about it, how they feel about it, and whether or not you have an organic community out there on the web sharing your documentation.

Beyond documentation data

I hope that this detail has given you a better understanding of different types of data, beyond documentation data, that are available to you as a technical writer to draw valuable conclusions from. By analyzing these types of data, you are prepared for prioritizing your documentation task list, but also better able to understand the customers of your product and documentation. Even if only some of these are available to you, I hope they are useful. Be sure to read Just Add Data: Using data to prioritize your documentationfor the full explanation of how to use data in this way.

Just Add Data: Using data to prioritize your documentation

Tue, 21 May 2019 17:15:40 +0000

This is a blog post adaptation of a talk I gave at Write the Docs Portland on May 21, 2019. The talk was livestreamed and recorded, and you can view the recording on YouTube: Just Add Data: Make it easier to prioritize your documentation - Sarah Moir

Prioritizing documentation is hard. How do you decide what to work on if there isn’t a deadline looming? How do you decide what not to work on when your list of work just keeps growing? How do you identify what new content you might want to add to your documentation?

By adding data to the process, it’s possible to prioritize your documentation tasks with confidence!

Prioritizing without data

Prioritizing a backlog without data can involve asking yourself some questions, like what will take the least amount of time? Or, what did someone most recently request? If I’m doing this, I might ask my product manager what to work on, or do whatever task seems easiest at the time. I might even focus on whichever task I can complete without talking to other people, because I’m tired.

Based on the answers to those questions, I’ll end up with a prioritized backlog, but lack confidence that what I’ve chosen to work on will actually bring the most value to customers and the documentation. Especially if I’m choosing not to do work, it can be a challenge to keep ignoring an item in the backlog because it doesn’t fit with what I think I need to be working on, especially without some sort of “proof” that it’s okay to ignore. To make this process easier, I add data.

Why prioritize with data?

Using data to prioritize a documentation backlog can help give you more confidence in your decisions and help you justify why you’re not working on something. It can challenge your assumptions about what you should be working on, or validate them. Adding data can help improve your overall understanding of how customers are using your product and the documentation, leading to benefits beyond the backlog.

Data types for prioritization

What kinds of data am I talking about? All kinds of data! If you skim the following list, you’ll notice that this data goes beyond quantitative sources. When I talk about data, I’m including all kinds of information: qualitative comments, usage metrics, metadata, website access logs, survey results, database records, all of these and more fit in with my definition of data. Here’s the full list

User research reports
Support cases
Forum threads and questions
Product usage metrics
Search strings
Tags on bugs or issues
Education/training course content and questions
Customer satisfaction survey
Documentation feedback
Site metrics
Text analysis metrics
Download/last accessed numbers
Topic type metrics
Topic metadata
Contribution data
Social media analytics

Some of these data types are more relevant to different types of organizations and documentation installations. For example, open source projects might have more useful issue tags, or organizations that use DITA will have easier access to topic type information.

This list of data types is to demonstrate the different types of information that can help you prioritize documentation, but I don’t want you to think that you need to do large-scale collections or implementations to get any valuable data worth incorporating into your prioritization process.

I’ll cover a couple of these data types in more detail here, but I talk about all of them in another post: Detailed data types you can use for documentation prioritization.

Product usage data

You can use usage data for products (also called telemetry) to find out where people are spending their time. What features or functionality are they using? Even if they’ve purchased or installed the product, are they actually using it?

Some examples of product usage data include:

Time in product
Intra-product clicks
Types of data ingested
Types of content created (e.g., dashboards, playlists)
Amount of content created (e.g., dashboards, playlists)

In addition to data about how people are interacting with the product, you can also gather product usage data without actual introspection into how people are using it. If you have information about how many people have downloaded a product or are logging in to a service:

Number of downloads and installs
License activations and types
Daily and monthly active users

I mostly talk about using data to help you prioritize the more ambiguous parts of a backlog that might not be tied to a release, but especially with the help of product usage data, you can better-prioritize release-focused documentation as well. If your product is in beta, and you want more data to help you prioritize your overall documentation backlog, you can use some product usage data to understand where people are spending more of their time, and draw conclusions about what to spend more time on or less time on, or what level of detail to include in the documentation, to achieve your overall documentation goals for the release.

Site metrics

Site metrics like page views, session data, HTTP referer data, and link clicks can help you understand where people are coming to your docs from, how long they’re staying on the page, how many readers there are, and what they’re doing after they get to a topic. Here are some example site metrics:

Page views
Session data like time on page
Referer data
Link clicks
Button clicks
Bounce rate
Client IP

You can also use this data to understand better how people interact with your documentation, like whether they’re using a version switcher on your page or expanding/collapsing more information hidden on the page.

You can also split this data by IP address to understand groups of topics that specific users are clustering around, to better understand how people use the documentation.

Identify questions based on your backlog

The process of adding data to your documentation prioritization strategy is all about making do with what you have to answer what you want to know. What you want to know depends on your backlog.

Data analysis is focused on a goal. You don’t want to collect a lot of data and then just stare at it, or get stressed out by the amount of “insights” that you could be gathering but meanwhile you’re not really sure what to do with the information. If you consider questions that you want to answer in advance, you can focus your data collection and analysis in a more valuable way.

Some example questions that you might identify based on your task list:

What are people looking for? Are they finding what they’re looking for?
Are people looking for information about /<thing I’ve been told to document/>?
What do people want more help with?
What people are we targeting that don’t see their use cases represented?

Tie questions to data types

After you’ve identified questions relevant to your task list, you can tie those questions to data types that can help you answer the questions.

For example, the question: What are people looking for and not finding?

To answer this, you can look where people are looking for information, namely search keywords that they’re typing into search engines, common questions being posted on forums, or the topics of support cases filed by customers.

For example, I looked at some data and was able to identify specific search terms people are using on the documentation site that are routing customers to a company-managed forum site. I can then use that data to identify cases where people are looking for documentation about something, but are not finding the answers in the documentation.

Another example question: What do people want more help with?

This could be answered by looking at the topics of support cases again, but also the types of questions being asked in training courses, as well as unanswered questions on forums.

As a final example: What market groups are we targeting that don’t see their use cases represented?

To answer this, you could look at data about sales leads, questions being asked by the field that contain specific use cases for various market verticals, as well as questions being asked in training courses.

Find questions from data

If you don’t have much of a task list to work with, or if you aren’t able to get access to data that can help you answer your questions, you can still make use of the data that is available to you and draw valuable insights from it.

You can identify interest in content that you maybe weren’t aware of, and make plans to write more to address that interest, or modify existing content to address that interest. Maybe there are a bunch of forum threads about how to do something, but nothing authoritative in the documentation. That information hasn’t made it to the docs writers in any way, but because you’re looking at the available data, you’re able to see that it’s important.

Even if you have no data specifically relevant to the documentation or customer questions, you can still find ways to identify documentation work to add to a task list. You could create datasets by performing text analysis on all or specific documentation topics, and identify complexity issues, or topics that don’t adhere to a style guide. You could use customer satisfaction surveys to identify places where documentation architecture or linking strategies could be improved.

Working with the data

Now you hopefully have a better understanding of different types of data available to you, and how you can identify valuable data sources based on your questions that you want to answer. But how much data do you need to collect? And how do you get the data? Most importantly, how do you analyze it to answer the questions you want to answer?

How much data?

How much data do you need to collect? You don’t need to collect data forever. You don’t need ALL the data. You just need enough data to point you in a direction and reduce uncertainty.

You can use a small sample of users, or a small sample of time, so long as it helps you answer your question and reduce uncertainty about what the answer could be. Collecting larger amounts of data doesn’t mean that you reduce uncertainty by an equally large degree. The amount of data you collect doesn’t correlate directly to what you’re able to learn from it. However, if the question you’re trying to answer with data concerns all the documentation users over a long period of time, you will be collecting more data than if you just want to know what a specific subset of readers found interesting on a Friday afternoon.

Try for representative samples that are relevant for the questions you’re trying to answer. If you can’t get representative data, try for a random sample. If you can’t get representative or random samples, acknowledge the bias that is inherent in the data you’re using. Add context to the data wherever possible, especially about who the data represents and why the data is still valuable if it isn’t representative.

You might find that collecting a small amount of data leaves you with more questions than answers, and that’s okay too. It’s an opportunity to continue exploring and learning more about your customers and your documentation tasks. But how do you even get any data at all?

How do you get the data?

You’ll either be collecting your own data, or asking others for the data you need.

If it’s data about the documentation site or its content, you might own that data yourself, and already have access to it. If it’s other types of data, like sales leads or user research data, it’s time to talk to the departments or people that manage those areas.

A business development department might have reporting on internal tools like sales leads or support cases.
Product managers can share direct customer data and product usage data if you don’t have direct access.
Project managers can share data related to internal development processes.

The teams managing different datasets will vary at your organization, and might even be you in many cases. They may be reluctant to share data. With that in mind, remember that when you collect data, you don’t need to get persistent access to all the data you want. Focus on getting some access to some data that is useful to answer your questions. After that, you can use that data to make your work more efficient and informed, and then hopefully communicate that value and get more access to data in the future if you want.

What to use for data analysis?

What do you use to analyze that data after you get it? How do you transform data into a report of useful information?

Some tools might already have analytics and reporting built in, like Google Analytics. That can certainly make it easier to analyze the data!

For other types of data that you need to analyze yourself, use the tools available to you. Think about what already know how to use, or have access to:

Know how to use Excel? Perfect! Get started collecting and processing data in spreadsheets and with macros.
Know how to write scripts in R/Python to analyze data? Great! You can write scripts to collect, process, and visualize this data.
Is your organization using a tool like Splunk, ElasticSearch, Tableau, etc.? Good news! You are really ready for data analysis.

You don’t have to spend a long time learning a new tool to analyze data for these purposes. If you continue incorporating data analysis into your work, it might make sense, but it isn’t necessary to get started.

Tools aren’t magic

It’s also important to note that tools aren’t magic. Some degree of data analysis will involve manual collecting, categorizing, or cleaning of the data. If your organization doesn’t have strict topic types, you might need to perform manual topic-typing. If you want to analyze some information but the data isn’t in a machine-readable format, you might have to sit at your desk copy pasting for hours.

Depending on your skills, the current state of the data that you want to analyze, and the tools available to you, the amount of time it takes to analyze data and get results can vary widely. I have spent 3 days manually processing data in Excel, and I’ve spent 2 hours creating searches in mostly-clean datasets in Splunk to get answers to various questions. Keep that in mind when you’re analyzing data.

How to perform data analysis

When you analyze data, what are you actually looking at?

Top, rare, outlying values

Find out what values are most common, and which values are least common. Those can be established by counting the various instances of values.

Look for values that are different from the others by a large margin. You can use standard deviation as a function to achieve this.

Patterns and clusters across data

You can also look for patterns and clusters in your data.

If you’re working with qualitative data, you might need to categorize, or code, the data so that you can sort it and look for patterns in the results. You can identify these patterns by counting instances of categories, or looking at clusters of behavior. An example of a cluster of behavior is if you look at documentation topic visits over time, and you identify a spike in visits at a particular time.

Split by different features

You also want to segment data by different features. Meaning, you can better understand the most common values if you split them by other types of information. For example, you can look at the most commonly visited topics in your documentation set over the last 3 months, or you can look at the most commonly visited topics in your documentation over the last 3 months, but on a week-to-week basis. That additional split can help you understand how those values are changing over time. If you identify a spike in a particular topic or category of topics, you can then interpret the data. Maybe a new product release led to a spike of interest in the release notes topic that wasn’t easily identified until you split the results by week. This is also a good opportunity to point out to a product team that people really do read your documentation!

That’s an example of splitting by time, but you can split by any other field available to you in your data. To use the same data type, looking at the most common topics by product, by IP address, or other factors, can help lead to valuable insights.

Combine data types

You can combine different types of data to understand approximately how many people are using the product vs how many of them are using the documentation. Comparing sales leads, product usage data, and existing page views could help you approximate the number of potential, and existing customers, alongside the number of distinct documentation readers.

Make sure that when you combine data across datasets, you keep track of units and time ranges, and make sure that you compare like data with like data. For example, be careful not to use data that refers to potential customers with data that refers to existing customers, because that could lead to misleading results if you don’t keep context with the data.

Interpreting results

When you interpret the results of your data analysis, make sure that you are adding context to the data. Especially when dealing with outlier data, but even when reviewing data like rarely-viewed or frequently-viewed topics, keep in mind additional context that could explain results.

Add context from expertise

Use your expertise and knowledge of the documentation to add context. For example, topics concerning a specific functionality are likely to be more popular at a specific time if that functionality was recently changed.

Pursue alternate explanations

Whenever you’re interpreting data, you want to make sure that you’re gut-checking it against what you already know. So if a relatively mundane topic has wildly out-of-the-ordinary page views, there are likely alternate explanations for that interest. Maybe your topic ended up being a great resource about cron syntax in general, even for people that don’t use your product.

Draw realistic conclusions

Draw realistic conclusions based on the data available to you. You might not be able to get access to or combine specific datasets due to privacy concerns. If you carefully identify what problems you’re trying to solve, and select only the data sources that can help you solve those problems, you can reduce the potential that you’ll introduce bias into your data analysis, and improve the conclusions that you’re able to draw.

Don’t trust data blindly

Don’t trust the data blindly. When reviewing data that seems out of the ordinary or like outliers, examine the different reasons why the data could be like that. Who does the data represent? What does it represent? Make sure that you’re interpreting data in context, so that you’re able to understand exactly what it represents. It can be tempting to ignore data that doesn’t match your biases or expectations.

Above all, remember to use data to complement your research and writing, and validate or challenge assumptions about your audience.

Your turn to add data

Identify the questions you’re trying to answer
Use the data available to you
Use the tools available to you
Analyze and interpret the data
Take action and prioritize accordingly

Additional resources

Find me on Twitter @smorewithface.
The Product is Docs book chapter on Measuring Success
How to Measure Anything book by Douglas Hubbard
Bob Watson’s posts on docsbydesign.com about measuring value
Just Enough Research book by Erika Hall
Nielsen Norman Group about handling small sample sizes.

So you want to be a technical writer

Tue, 05 Feb 2019 17:12:59 +0000

If you’re interested in becoming a technical writer, or are new to the field and want to deepen your skills and awareness of the field, this blog post is for you.

What do technical writers actually do?

Technical writers can do a lot of different things! People in technical writing write how-to documentation, craft API reference documentation, create tutorials, even provide user-facing text strings to engineers.

Ultimately, technical writers:

Research to learn more about what they are documenting.
Perform testing to verify that their documentation is accurate and validate assumptions about the product.
Write words that help readers achieve specific learning objectives and that capture what the writer has learned in the research and testing processes.
Initiate reviews with engineers, product managers, user experience designers, quality assurance testers, and others to validate the accuracy, relevancy, and utility of the content.
Advocate for the customer or whoever uses the product or service being documented.

The people reading what technical writers have produced could be using software they’ve purchased from your company, evaluating a product or service they are considering purchasing, undergoing a required process controlled by your organization, writing code that interfaces with your services, configuring or installing modifying hardware produced by your company, or even reviewing the documentation for compliance and certification purposes. Your goal, if you choose to accept it, is to help them get the information they need and get back to work as soon as possible.

Identify what you want from your career

Some general career-assessment tips:

Identify what motivates you and what challenges you.
Identify what type of team environment you want. These are loose descriptions of types of team environments that are out there:
- A large highly-collaborative team with lots of interaction
- A distributed team that is available for questions and brainstorming as needed, but largely everyone is working on their own thing.
- A small team that collaborates as needed.
- A team of one, it’s just you, you are the team.

Is technical writing a good fit for you?

Do you enjoy explaining things to other people?
Do people frequently ask you to help explain something to them?
Do people frequently ask you to help them revise content for them?
Do you care or enjoy thinking about how to communicate information?
Do you identify when things are inconsistent or unclear and ask people to fix it? (Such as in a UI implementation, or when reviewing a pull request)
Do you enjoy problem-solving and communication?
Do you like synthesizing information from disparate sources, from people to product to code to internal documentation?
Do you enjoy writing?

My background and introduction to technical writing

I started in technical support. In college I worked in desktop support for the university, wandering around campus or in the IT shop, repairing printers, recovering data from dying hard drives, running virus scans, and updating software. After graduation I eventually found a temp job working phone support with University of Michigan, managing to turn that position into a full-time permanent role and taking on two different queues of calls and emails.

However, after a year I realized that was super exhausting to me. I couldn’t handle being “on” all day, and I found myself enjoying writing the knowledge base articles that would record solutions for common customer calls. I wrote fifty of them by the time I discovered a posting for an associate-level documentation specialist.

I managed to get that position, and transferred over to work with a fantastic mentor that taught me a ton about writing and communicating. After a few years in that position, writing everything from communication plans (and the accompanying communications), technical documentation, as well as a couple video scripts, I chose to move to California.

With that came another set of job hunting, and realizing that there are a lot of different job titles that technical writing can fall under: UI writer, UI copywriter, technical writer, documentation specialist, information developer… I set up job alerts, and ended up applying, interviewing, and accepting an offer for a technical writing position at Splunk. I’ve been at Splunk for several years now, and recently returned to the documentation team after spending nearly a year working in product management.

Where people commonly go to technical writing from

Technical writers can get their start anywhere! Some people become technical writers right out of college, but others transition to it after their career has already begun.

As a technical writer, your college degrees doesn’t need to be in technical writing, or even a technical-specific or writing-specific field. I studied international studies, and I’ve worked with colleagues that have studied astronomy, music, or statistics. Others have computer science or technical communication degrees, but it’s not a requirement.

For people transitioning from other careers, here are some common starting careers:

Software developers
UX practitioners
Technical support

That’s obviously a short list, but again if you care about the user and communication in your current role, that background will help you immensely in a technical writing position.

Prepare for a technical writing interview

Prepare a portfolio of writing samples

Every hiring manager wants to see a collection of writing samples that demonstrate how you write. If you don’t work in technical writing yet, you might not have any. Instead, you can use:

Contributions you’ve made to open source project documentation. For example, commits to update a README: https://github.com/yahoo/gryffin/pull/1
How-to processes you’ve written. For example, instructions for performing a code review or a design review.
A blog post about a technical topic that you are familiar with. For example, a post about a newly-discovered functionality in CSS.
Basic task documentation about software that you use. For example, write up a sample task for how to create a greeting card in Hallmark Card Studio.

Your portfolio of writing samples demonstrates to hiring managers that you have writing skills, but also that you consider how you organize content, how you write for a specific audience, and the level of detail that you include based on that audience. The samples that you use don’t have to be hosted on a personal website and branded accordingly. The important thing is to have something to show to hiring managers.

Depending on the interviewer, you might perform a writing exercise in-person or as part of the screening process. If you don’t have examples of writing like this, that’s a good reason to track down some open source projects in need of some documentation assistance!

Learn about the organization and documentation

Going in to the interview, make sure you are familiar with the organization and its documentation.

Read up about the organization or company that you are interviewing with. If you can, track down a mission statement for the organization.
Find the different types of documentation available online, if possible, and read through it to get a feel for what the team might be publishing.
If the organization provides a service or product that you’re able to start using right away, do that!

All of these steps help you better understand how the organization works, what the team you might be working on is producing, and demonstrates to the interviewer that you are motivated to understand what the role and the organization are about. Not to mention, this makes it clear that you have some of the necessary skills a technical writer needs when it comes to information-gathering.

Questions you might want to ask

Find out some basic team characteristics:

How many other technical writers are at the organization?
What org are the technical writers part of?
Is there a central documentation team or are the writers scattered across the organization?
How distributed is the documentation team and/or the employees at the organization?

Learn about the documentation process and structure:

What does the information-development process look like for the documentation? Does it follow semi-Agile methods and get written and researched as part of the development team, or does information creation follow a more waterfall style, where writers are delivered a finished product and expected to document it? Or is it something else entirely?
Are there editors or a style guide?
Do the writers work directly with the teams developing the product or service?
What sort of content management system (CMS) is in use? Is it structured authoring? A static-site generator reliant on documentation files written in markdown stored next to the code? A wiki? Something else?

Find out how valuable documentation is to the organization:

Do engineers consider documentation vital to the success of the product or service?
Do product managers?
Do you get customer feedback about your documentation?
What is the goal of documentation for the organization?

Some resources for getting started with technical writing

Books to read

These books cover technical writing principles, as well as user design principles. None of these links are affiliate links, and the proceeds of the book I helped author go to charity.

The Product is Docs by Christopher Gales and the Splunk documentation team
- Yes, I helped.
Every Page is Page One by Mark Baker
- This book is a great introduction and framework for writing documentation for the web.
Developing Quality Technical Information by Michelle Carey, Moira McFadden Lanyi, Deirdre Longo, Eric Radzinski, Shannon Rouiller, and Elizabeth Wilde.
- This book is a great resource and reference for detailed writing guidance, as well as information architecture.
Design of Everyday Things by Don Norman
- The classic design book covers user-focused principles that are crucial to writing good documentation.

This is an intentionally short list featuring books I’ve found especially useful. You can also consider reading Scenario-Focused Engineering: A toolbox for innovation and customer-centricity, Nicely Said: Writing for the Web with Style and Purpose, Content Everywhere: Strategy and Structure for Future-Ready Content, Design for How People Learn, and Made to Stick: Why Some Ideas Survive and Others Die.

Articles and blogs about technical writing

I like following resources in RSS feeds to get introduced to good thinking about technical writing, but not all good content is new content! Some great articles that have helped me a lot:

Blogs to follow (intermittently updated)

Beth Aitman’s http://uiwriting.tumblr.com/
Mark Baker’s http://everypageispageone.com/
Tom Johnson’s http://idratherbewriting.com/
Nielsen Norman Group articles: https://www.nngroup.com/articles/

Great articles about technical writing

How Chunking Helps Content Processing by Kate Moran for Nielsen Norman Group
Writing great documentation by Taylor Singletary
Users’ Advocate: Balancing Just-in-Time Support Docs and Customer Experience by Neal Kaplan (who I now work with)
How can technical writers cut through engineering jargon and decode complex information? by Tom Johnson
Top 10 Most Frequently Asked Questions about Technical Writing by Tom Johnson
A little thing about release notes by the Slack team
How to write a great error message by Thomas Fuchs
What is the technical writer’s role in content marketing? by Tom Johnson
Writing truthful documentation by Tom MacWright
Inverted Pyramid: Writing for Comprehension by Amy Schade for Nielsen Norman Group
Plain Language Is for Everyone, Even Experts by Hoa Loranger for Nielsen Norman Group
Break Grammar Rules on Websites for Clarity by Hoa Loranger for Nielsen Norman Group

Other web resources

Twitter is a great resource for building a network of people that care about documentation. If you use it, I recommend searching for people who commonly tweet with #writethedocs.

Write the Docs is a conference and community founded by Eric Holscher and maintained by a brilliant set of volunteers!

The Write the Docs Slack workspace is fairly active, and includes channels for job postings, career advice, as well as current discussions about trends and challenges in the technical writing world.

Some talks from the conference I recommend checking out are visible on YouTube:

Write the Docs Portland 2017: Error Messages: Being Humble, Human, and Helpful… by Kate Voss
Write the Docs Portland 2017: Caring Systems: Documentation as care by Amelia Abreu
Write the Docs Portland 2017: Even Naming This Talk Is Hard by Ruthie BenDor
Write the Docs Portland 2017: Start with the tasks, not the endpoints by Sarah Hersh
Write the Docs Portland 2016: Write the Readable README by Daniel Beck
Write the Docs Portland 2016: Copy That: Helping your Users Succeed with Effective Product Copy by Sarah Day
Write the Docs Portland 2016: 7 Values of Effective Tech Writing Teams by Joao Fernandes
Write the Docs Portland 2016: Oops, I Became an Engineer by Tara Scherner de la Fuente

There are playlists for 2018 (which I did not attend) and earlier years as well on YouTube, so dig around there and find some more resources too if watching videos is useful to you!

Feature Names Matter

Fri, 11 Aug 2017 11:28:35 +0000

When someone starts using your software, they need to build an understanding of how it works and how the pieces interact. The UI text you write and the feature names you choose can build or break a mental model.

From a marketing perspective, the importance of the name is clear. You want something catchy, marketable, searchable, memorable, all these things. But most importantly, a feature name must help a user build a mental model of what your feature does.

The mental model helps the user understand why they might use this feature, and what for. One of the riskiest part of shipping something new is adoption. If people don’t know what it does or how it works, they won’t use it. A crucial element to that understanding is what you call the new thing and how you describe it in the product. If I can’t guess based on the name what it does, I might not click on it at all and explore it.

Let’s look at some feature names…

Google+ vs GoogleDocs. One of these is pretty opaque, and the other is pretty clear. I might think that Google Docs is google FOR docs, but as soon as I click into it, I’ll see what it is and understand that it’s for writing docs. I might never click into Google+ because I have no idea what it is based on the name.
Dropbox vs Box. There’s a reason both of these companies are named practically the same thing. Because you put things in boxes that you want to share and store. It’s a super evocative mental model, so it gets a bit overused, perhaps.
Slack vs HipChat. HipChat is a bit more descriptive, but you know automatically that it’s a chat app. Slack turns a verb into a noun, and hopes that you start using it and understand that you slack off while using it… kind of.

It’s harder to come up with examples in software of things that truly failed, because they aren’t very well known. But the example that brought this to life for me is from a card game I learned how to play recently. Red7 uses the concept of a “canvas” and a “palette” to tie the metaphor of color across the game. But combining those concepts with the established mental model that you have in a card game with a discard pile and a hand of cards took quite a bit of work. In reality, the clever metaphor broke down and impeded what could have been quick understanding by burdening an existing card game mental model with a mental model of painting ephemera. It was marketable, but not intuitive because it didn’t help people build a mental model to understand how the game works.

The simplest way to pick a good feature name is to test them out. Do some word association exercises with your team, but also with people that don’t work on your team and don’t even work in software. Diverse teams matter a lot in this exercise. This can help identify names that build mental models, break them, or are irrevocably associated with irrelevant mental models.

Another way to pick good feature names is to rely on scenarios when building features. That way, you’re less likely to conceptualize a feature based on its architecture, or your internal team structures, and more likely to think of it from a problem-solving perspective. If you know exactly what the feature is doing, and for whom, it’s easier to pick a useful name.

tweetthedocs: Use Twitter to meet your users where they are

Tue, 31 May 2016 13:30:50 +0000

As a tech writer, it’s hard to tell how users get to your docs at all. They might be clicking on in-product help links, searching the web, or getting sent links from support. But you can get proactive about it too. Help users of your product get their questions answered by meeting them where they are—on social media sites like Twitter. You may already rely on marketing, sales, support, and search engines to bring users to your documentation, but social media is a direct option. You can tweet about anything from general topics that answer common user questions to drier topics that are important for people to know. Read on to learn how!

Why to try tweeting your documentation

Using a content marketing approach like tweeting can help you reach users with your docs—and thinking from the perspective of tweetable product documentation can help you write more approachable, plain language and even improve the topic headings that you may be using.

Short sentences are easier for people to understand, but they aren’t always easy to write. You may end up explaining things in long, complex sentences because you’re still trying to grasp the topic yourself. If you struggle to come up with pointers or teasers to your docs, you may want to rewrite them to keep readers reading.

Restructure your sentences by thinking about how you might phrase a lead-in from Twitter. One-liners that describe a problem (or part of one) help people understand what they’ll get if they click—or keep reading.

“Here’s what’s new in this release” vs “Read this before you upgrade”
“About this dashboard” vs “Detect suspicious activity with this dashboard”
“Using this product” vs “Improve your security posture with this product”

Collect tweetable candidate docs

Google helps your users if they know what to look for, but Twitter can help if they don’t know where to start. Good topics to highlight on Twitter are those that answer common user questions, topics that may be dry but are important for people to know, tips and tricks that may be buried in a longer topic, and getting started information.

Examine the topics that aren’t getting much attention using site analytics. After you assess the writing quality and the location of each topic, determine if the information is useful (if it isn’t, that could explain the lack of hits). If the topic has a small, but essential use case, it could be that people who need it might not know it’s there.
Examine the topics that are getting a lot of attention using site analytics. Just like the lonely topics, the popular topics are great candidates for sharing. Docs with a lot of hits tend to contain vital information—so the more people that see them, the better.
Discover common struggles and use cases for your product with help from support. Your support team is your best ally—talk to them to see what customers ask about often, and what they’re trying to do with your product or service. You can answer those questions and address those goals proactively by sending your docs into the twittersphere.
Unearth hidden tips and tricks. Small paragraphs can hide important knowledge relevant to specific use cases. If someone is reading a topic for a specific goal, they may miss information that is good-to-know but not relevant in the moment. Highlight those bits of information with a tweet.

Plan your tweets

Once you collect a set of topics that are good to share, it’s time to get ready to start tweeting. If you have a marketing or communications team, work with them. They’ll help you avoid common pitfalls of social media marketing and communication, set up a content calendar, and define a voice that works with the company’s goals. Many of the steps for tweet planning are the same that you already use for doc planning.

Get an account. If you don’t have a company Twitter account to use, start one. Get permission to the main company one, or start a documentation-specific Twitter.
Schedule your tweets. Use a service that lets you schedule tweets. This lets you stay focused on your work, and tweet strategically instead of sporadically.
Know when to start. Pay attention to important times in your industry. A company-sponsored conference is a great time to start tweeting—your biggest fans will be there and ready to learn and spread the word after the conference is over. If your company isn’t large enough to have its own conference, glom on to another big industry conference, or just start tweeting!
Find your voice. Voice is important in writing, and even more so on Twitter. Treat the tweets like your docs—don’t patronize and don’t make assumptions. Evoke professionalism and trustworthiness. Be careful to use a voice that will resonate with the social media users—don’t sound too stilted, too corporate, but also not too friendly.
Define your audience. Determine which portion of your customer base is likely to be on Twitter. See what your competitors are doing.
Plan your frequency. If you’re joining an existing Twitter account, pay attention to how often that account tweets so that you don’t overwhelm the existing content. Post often enough to make following the feed valuable to readers.
Time it right. Decide what time of day your tweets should go out. If your audience is all in one location, pick times when you think they’ll be online. If you have a global audience, keep that in mind and plan to target specific segments of your audience at specific times of day.

Start tweeting

Start writing your tweets!

Write a backlog, and plan meetings in the future to refill it. This is the kind of backlog you want to have—a backlog of content to put on the internet. Write up 20-30 tweets before you start tweeting to give you a good start. Schedule meetings in the future with yourself and anyone else helping you out to write new ones—checking the analytics to see if specific times, verbiage, or topics resonated with your audience, and try new ideas too!
Include graphics to encourage engagement. Include diagrams and screenshots to help your tweets stand out. Don’t include them in every tweet, and not just for the sake of it, but rather when they’re useful.

After you start tweeting

Pay attention to what’s happening on Twitter.

Get feedback. Have regular meetings with the people who manage the Twitter account (if it isn’t you) to hear about how your tweets are doing, and learn what topics to prioritize in the future
Watch the feed. Be cautious around events or news that dominate social media posts for awhile. It’s obvious when you’re using a scheduling service if your tweets about how to use your product show up at the same time everyone is livetweeting a political debate or reacting to a tragedy in the news. On the other hand, stay attentive to trends that may be relevant to your product. If you make a security product, a security vulnerability or breach may be a good time to tweet more often.
Monitor the feed. Make sure someone is dedicated to monitoring the feed and addressing replies from customers and potential customers (and the spambots and bullies).

Tweeting the docs

Improve your documentation and collaborate with other teams to #tweetthedocs. Write better sentences and headers, without sounding too much like a marketing #brand that you alienate your readers. Don’t wait for Google to bring customers to your docs—reach out to them proactively!

Affective Computing and Adaptive Help

Wed, 14 Oct 2015 10:06:00 +0000

Several months ago, I saw Dr. Rosalind Picard give a talk on Affective Computing. I took notes and thought a lot about what she said but let my thoughts fester rather than follow up on them. Then last week, I read Emotional Design by Donald A. Norman, which reminded me of Dr. Picard’s work and my initial thoughts about affective computing. There are two elements to affective computing:

People interact with technology and devices as though it has a personality (and devices and interfaces without personalities can be distasteful to use).
Cameras, wearables, and other technology can be used to determine the emotions and affective responses of a person using technology with surprising accuracy.

Websites and applications are personalized by tracking your browsing history, collecting advertising preferences, device usage, and demographic data. Using affective computing, they could soon be personalized by tracking your emotions.

Help tailored to you

The great challenge of online or in-product help is providing help you want, when you need it and just-in-time help, offering guidance before you get frustrated with the experience. This is commonly referred to as adaptive content or adaptive help. Using advances in affective computing, adaptive help could be customized not only based on your knowledge of a product, what device you’re using, and where you are located, but also based on the emotions you experience as you use the product. By identifying capability personas, it’s easier to understand what types of help certain types of users might look for, and what kind of help they might need.

Have you logged in and performed x number of actions over the last week, and each week you confidently perform at least one new action successfully? Congrats, you’re now a part of the “confident autonomy” group.
Have you logged in once a week and performed the same action every time, occasionally flitting to other views within the application but never completing anything but that one action? Congrats, you’re part of the “methodical novice” group.

As a member of the confident autonomy group, you might look for suggestions about what types of actions to learn next, while as a methodical novice, you might want to know about process improvements or how to expand your use of the tool. Knowing when to introduce that help is also key, and this is where the inclusion of affective computing could be vital. You might want help right when they start to get frustrated, indicated by your forehead wrinkling in irritation. Or perhaps if you click through the beginning of a process repeatedly, but never complete it, looking confused all the while. Successfully addressing these scenarios is challenging. It’s hard to understand and anticipate the “why” behind someone’s emotions and actions. We can’t know what is frustrating someone. We can’t always find out what is stopping someone from completing a process.

A second iteration of Clippy?

Microsoft tried to anticipate the “why” behind people’s actions in a rudimentary and now-notorious implementation named “Clippy,” designed to respond when it noticed certain patterns in the Microsoft Office. Start a paragraph with “Dear Susan,” and Clippy would appear and ask if you were writing a letter, and if so, could it be of any help? The capability now exists to make a Clippy-like tool far smarter using machine learning and affective computing. This tool could take in a large amount of data before making suggestions.

No longer as simple as recognizing a structure of “Dear Susan,” the tool could make note of how many times an account has written Dear Susan before. For people who write letters often, the tool could “learn” that they probably don’t need assistance writing a letter. For someone who has never written “Dear Susan” before, the tool could take that into account before offering advice.

Using affective computing, the tool could monitor your facial expressions, or sync up with a wearable device, to assess your emotions as you engage with a product. If you’re bored, excited, or confused, Bindy could offer varying suggestions. Truly adaptive help could be responsive to each of these signals, changing the help message accordingly. Rather than ask if you needs help writing the letter, the tool could ask instead if you need help writing the letter faster, by pointing to some templates. the tool could also suggest help for sending a letter to multiple people with mail merge tips. If the name matches someone in the user’s contact lists, the tool could automatically suggest (a la Crystal) ways to better target their audience.

Surveilling you to help you

The obvious challenge in this day and age is that the technology required to make this work requires near-pervasive tracking—a usually unwelcome act of surveillance. Software to observe and monitor your emotions, actions that you perform in a product and on a device, that integrates with your contacts and friends lists would require you to place a lot of trust in a product or service and its ability to protect your privacy.

However, this isn’t too far off from what currently exists in many of our devices. As I draft this in Google Docs, my writing history is tracked. Google knows exactly how many times I have (or have not) written “Dear Susan” and similar formulations. Google also has access to my contact lists, data on how often I contact them, and a convenient way to set up a video feed of my face integrated in Google Hangouts and Google Mail. The new element of the tracking you to provide you with a personalized experience is affective computing. Currently, the most effective form of affective computing requires a webcam or another way to track your facial expressions. Other types of affective computing rely on voice-based analysis. But as wearable devices become more common, and integrate heart rate monitors and the movement of your wrists as you type, the potential for the integration of affective computing into a daily workflow becomes more realistic and subtle than a constant video feed of your face.

Adaptive help won’t stop butting in

In addition to the necessary surveillance, the biggest challenge to this entire concept is a challenge the Nielsen Norman Group identifies in all adaptive help and the reason why the original Clippy failed: “it appeared without the users’ direct request. Like pop-up windows, the animation surprised and annoyed users who hadn’t wanted help”. Clippy, or any potential replacement tool, would be sharing potentially helpful information in the least-receptive manner possible. Unasked-for, with no understanding of your thought processes, much like your parents when you were a teenager. Another risk is also one that strikes parents of teenagers, as the Nielsen Norman Group makes clear: ”If the system suggests topics that users aren’t interested in, they will quickly learn to disregard those suggestions.” These adaptive help systems would have to be finely attuned to their user’s help needs, and provide that information at just the right moment (or determine whether to provide it at all).

Challenges of “smarter” adaptive help

Scrunching up your face isn’t a request for help—it could be a sneeze (it’s allergy season) or it could be the frustration right before you make a triumphant leap in your knowledge and understanding. Even the best-designed adaptive help system, using machine learning and affective computing, faces a near-insurmountable challenge. As Donald A. Norman reminds us in Emotional Design, “The proper response to an emotion clearly depends upon the situation.” The proper response to frustration isn’t always immediate help, and understanding the cause of the frustration is essential. A few examples from Norman:

“If a student is frustrated because the information provided is not clear or intelligible, then knowing about the frustration is important to the instructor [in this case, the adaptive help interface/intelligence], who presumably can correct the problem through further explanation.”

Incorporating an adaptive help response at this point would also not fix the true issue. If a person has gotten far enough only to be frustrated by unintelligible information, there is likely a user experience or interface issue at fault. Offering help is not the solution for every problem.

“If the frustration is due to the complexity of the problem, then the proper response of a teacher might be to do nothing. It is normal and proper for students to become frustrated when attempting to solve problems slightly beyond their ability, or to do something that as never been done before.”

Providing a helping hand here could, in some cases, stunt your growth as you become more comfortable with a software product, as you begin to rely not on your own experience with the product but instead lean on the adaptive help to tell you what to do. Norman continues, “if students aren’t occasionally frustrated, it probably is a bad thing—it means they aren’t taking enough risks, they aren’t pushing themselves sufficiently.” In the case of people that fit the proficient novice capability persona, this could be an opportunity for the adaptive help to push a person toward new ideas, concepts, or functionality that they may not have tried before. Another point of Norman’s is relevant for this specific frustrated student too: “It probably is good to reassure frustrated students, to explain that some amount of frustration is appropriate and even necessary.” Not all frustration is a problem to be fixed. When providing adaptive help, understanding the cause of frustrations is essential. Affective computing could help here.

“if it goes on too long, however, the frustration can lead students to give up, to decide that the problem is above their ability. Here is where it is necessary to offer advice, tutorial explanations, or other guidance.”

Using available data to help understand when a user might want help is the best use case for adaptive help. Norman’s explanations of the different types of appropriate reactions to frustration offer many cases where it may be inappropriate to offer help. The appearance of our Clippy replacement tool, whether it takes the form of a cleverly designed pop-up or slide-in modal, will be summarily rejected, along with its suggestions, if it appears at an inappropriate time. In contrast with many of the software development goals in the industry, adaptive help doesn’t leave much room for iteration. User trust is not easily won back once lost.

Give adaptive help feelings

To get it right the first time, we need to give adaptive help some feelings. An essential element of emotional design, identified by Norman in Emotional Design, is that the system has to have its own emotions as well, and the right kind for the situation. Style guides emphasize voice and tone for writing, and the same is true for adaptive help. Our tool would need a personality. Clippy made us feel stupid and wasn’t very smart itself, yet it was easy to personify—it even had eyes and a mouth. An ideal replacement tool would have a personality, necessarily targeted at the emotions of its users. Systems built without emotions are often nevertheless personified, so it is better to design for the inevitability than to design an emotionless help-providing system that feels cold to its users.

Let users ask for help

If you could predict when Clippy’s replacement would appear, or know how to reliably summon it when you knew you needed help, the tool could be more helpful. Chat robots already allow you to reach out and ask a question, though their subsequent helpfulness is debatable. You can summon slackbot in chat with a keyword or phrase, or ask it a question directly. Occasionally, the help menu in a software application lets you ask for help. More often, asking for help in the help menu sends you to a forum, the homepage of the documentation site, or a list of not-quite-relevant yet “frequently” asked questions. Making the appearance of Clippy’s replacement predictable or summonable makes it a friendlier help experience. Less of an unpredictable black box—say, someone that shows up and walks into your house when they want to—and more of a good friend that shows up when you call them, but also sometimes just when you need them, whether you called them or not.

Building a replacement Clippy

Adaptive help to this degree seems almost untenable. To build a help system this responsive and involved, you’d probably be better off investing the money in more customer support staff, designers, and technical writers. Building this type of tool would be technologically involved. You’d need several different elements:

Machine learning, to better characterize the habits and patterns of application users. Define who is confidently autonomous, who is a methodical novice, and who is an outlier.
Affective computing, to know to offer help when someone is frustrated, confused, or irritated with the application, and to understand how to frame the message. A frustrated person probably needs to hear a different message than a confused person.
Emotional design, to lend a friendly, understanding character to the help content.
Anticipatory computing, to take machine learning a step further and also allow a way to correct the help suggestions.

Building the actual help content that would power Clippy’s replacement would be even more challenging. Rather than writing content for the most common denominator, we would instead be writing for every possible denominator. While challenging, models exist for building adaptive content. And if done properly, adaptive content allows us to properly control the context that our content appears in. When we know the context, we can write properly understanding content.

The present state

An adaptive help tool like a replacement Clippy could be valuable, but simply not worth the investment. The risk of losing a person’s trust in an application or website, coupled with the complexity, surveillance, and gray areas involved in building such a tool make it impractical. However, each element that I outlined to structure a replacement Clippy exist in various application contexts. Machine learning is being used to drive anticipatory computing to tailor messages and notifications to you when you need them using data like your location, device, and how you use an application. For more on anticipatory computing, see How A Computer Can Anticipate Users’ Needs (Without Driving Them Crazy) in Fast Company, by Paul Montoy-Wilson. Kristen V. Brown sheds some light on other applications of machine learning in her piece for Fusion, I tried three apps that claim to make you more likable—and am now addicted to one of them.

Affective computing is primarily being used to gauge the effectiveness of advertising campaigns, ensuring that ads will continue to grow more targeted. See Raffi Khatchadourian’s piece in The New Yorker, We Know How You Feel. Andrew McStay discusses the wearable component of affective computing in The Conversation, Soon smartwatches will listen to your body to work out how you’re feeling, and much of Rosalind Picard’s groundbreaking work in affective computing involves wearables as well.

Nathan Collins discusses voice analysis and affective computing in his piece for Pacific Standard, You Sound Sad, Human. Emotional design has had an entire book devoted to it by Donald A. Norman. Beth Dean discusses Facebook’s efforts to incorporate Emotional Intelligence in Design on Medium, while Neil Savage discusses the personalities of robots in Artificial Emotions for Nautilus. Andrew Wilkinson compliments slackbot in his piece Slack’s $2.8 Billion Dollar Secret Sauce on Medium.

Adaptive content is discussed on Firehead’s blog by Noz Urbina in What is adaptive content? and Kate Sherwin assesses the design and user experience of adaptive help interfaces for Nielsen Norman Group in Pop-ups and Adaptive Help Get a Refresh. Especially important to consider from a design and content perspective are Sara Wachter-Boettcher’s words on Everybody Hurts: Content for Kindness.

Prescriptive Design and the Decline of Manuals

Tue, 24 Feb 2015 18:35:24 +0000

Instruction manuals, and instructions in general, are incredibly important. I could be biased, since part of my job involves writing instructions for systems, but really, they’re important! As this look into the historical importance of manuals makes clear, manuals (and instructions) make accessible professions, tools, and devices to anyone that can read them (which, admittedly, could be a hurdle of its own):

“With no established guild system in place for many of these new professions (printer, navigator, and so on), readers could, with the help of a manual, circumvent years of apprenticeship and change the course of their lives, at least in theory.”

However, as the economy and labor system shifted, manuals did too:

“in the 1980s, the manual began to change. Instead of growing, it began to shrink and even disappear. Instead of mastery, it promised competence.”

And nowadays, manuals are very rarely separate from the devices or systems they seek to explain:

“the help we once sought from a manual is now mostly embedded into the apps we use every day. It could also be crowdsourced, with users contributing Q&As or uploading how-to videos to YouTube, or it could programmed into a weak artificial intelligence such as Siri or Cortana.”

By attempting to make things more seamless, integrated, and easy-to-use, devices not only technologically lock out users with copyright and DRM and pre-soldered hardware, they also provide short, “quick start guides” in place of a true manual before you can begin using a machine.

“Furnished only with a manual of one or two pages, users soon reach a comfort zone, a knowledge plateau from which they tend not to wander. The aggregate effect, culturally, may be that less is less. The less we’re inclined to know about our devices, the more beholden we are to the manufacturers that make them, and the more we offer control to those who, for good or for ill, know more than we do. If manuals began as great equalizers, then their disappearance should at least give us pause. By dispensing with them, we could, consciously or no, be setting the stage for something few would relish: a society divided.”

This is certainly preferable by a large portion of people (judging by the types of calls and help requests I would get during my years of customer support). People want devices to just work, so they can move on to what they need to do–buy groceries, file their taxes, and yes, write documentation.

The trouble is that we’ve built a near-incomprehensible technological system, and now plan on using that technology in nearly every device on the market today. The internet of things is here, it doesn’t always work right (video), and the design isn’t usually easy to understand.

Design, when algorithmically-controlled to the point of being a total black box to a person, isn’t exactly helpful. In those cases we’re stuck wondering, Why is my house this temperature? Why is Facebook showing me this? Why am I being shown an ad about asphalt mixtures?

The rise of the algorithmically-sorted life isn’t exactly welcome, even if it purports to be helpful and make things easier to use (okay maybe the asphalt mixture ad is under no such illusions). However, that doesn’t necessarily make all prescriptive design bad (or do you think it does?). Some prescriptive design is basic and helpful–make it easy and clear what the person should do next.

Google has started using insistent, prescriptive design when it comes to various kinds of security warnings.

Gmail warns for suspicious email.
Search blocks “crapware” and malicious software download sites from appearing in search results.
Chrome blocks malicious software from being successfully downloaded at all.
Chrome improved its warnings for SSL (secure sockets layer). (Oddly, Wikipedia no longer has a dedicated page about SSL, instead directing people to the page for TLS (the encryption method that superseded it). The page now has a warning because it’s too long.)

Google is taking two different steps with these design choices.

In the case of the spam emails, when they are identified as malicious, Google tells you exactly why you should be suspicious of the message–perhaps recognizing that more information in this case could mean more obedience to best practices, e.g., don’t click on a phishing email.
In the case of the malicious software downloads and the SSL warnings, Google takes the step of making it hard for you to accomplish something (click through a warning, download a file, click on a search result) with the end goal of preventing you from doing something harmful (putting your data at risk in a man-in-the-middle attack, having your computer infected with malware or a virus).

This isn’t so bad, as long as the design choices are made with strict user benefit in mind (and to that end, who gets to decide what is beneficial for the user?) As the presentation on the SSL warnings makes clear, the goal is to change behavior, since changing comprehension alone didn’t warrant enough of a change.

Google may be parenting the Internet with their design choices, but in this case, the Internet could use a little bit of growing up (and so could its users).

Writing error messages is an art in and of itself. A guide on how to write error messages identifies that:

“A good error message has three parts:

The problem - explains that an error has happened;

The cause - explains what caused the problem;

The solution - explains how to overcome the problem.”

This is even tougher to do when you need to keep them as short as possible. Error messages today can be overly generic, and even purposely unhelpful in the hopes of making a system more secure. Two types of errors are especially guilty of this:

With a little bit of work, these workflows can be improved. System security can be strengthened without providing vague, unhelpful error messages to a person. I’ve attempted to log in 10 times in a row, getting the “username or password is incorrect” message and re-entering my password (since I can’t see my password when I enter it, it just shows up as little black dots). After 10 tries I finally realized that I had mistyped my email address on the first try, but never thought to correct it (it’s the shorter of the two, after all).

tech writing on This is important

Developing an AI strategy for documentation

Why you might need an AI strategy

Define your AI strategy

Partner with teams building AI into your product

Improve your content and content strategy

Expose your content to LLMs

Provide decision support and user-centric content

Write precise and clear content

Measure how people use AI to interact with your documentation

Evaluate the performance of AI tools when answering questions

Explore use cases for AI in your documentation practice

Implement your AI strategy

Save a horse, use a content strategy

Consider the context

Improve access and awareness

Are AI chatbots docs?

Add disclaimers

Strategically supplement responses

Test outputs

Monitor input and output

My writing setup

My VS Code setup

Code Spell Checker

Markdown All in One

Markdownlint

TODO Highlight

Indent-Rainbow

WordCounter

What I don’t use

Other VS Code settings

Google Docs settings

Other shortcuts for writing

A career bucket list for technical writers

Docs as code is a broken promise

What is docs as code?

The promise of docs as code

The pitfalls of docs as code

Git is confusing

Processes must be defined and reflected in Git workflows

Tools to write docs can be inconsistent

Merge gates and build checks are great… if you have them

Reviewing content in the repository is hard to read

Upholding the docs as code promise

What about GIFs instead of screenshots?

Displaying content as a graph: An exploration

What does it mean to display content as a graph?

Why hierarchies are common

Why use something different?

Serve multiple mental models

Write better documentation

Improve content reuse

Improve machine legibility

Why don’t we display content as a graph?

It’s difficult to design

It’s confusing for readers

It’s too flexible

It’s difficult to write chunked content

Sparse content and dense content don’t scale in a graph

Visible graphs: The future of displaying content?

Should you add screenshots to documentation?

Why screenshots are bad

Why screenshots are good

Identify the purpose of a screenshot

Make screenshots accessible

Use simplified screenshots

Automate screenshots

Intentionally add images to your documentation

Improving documentation findability in an age of low-quality search results

Get social: Invest in your community

Beyond the tooltip: Bring the docs to the product

Embrace AI: Chatbots and more

Improve documentation findability by going beyond the page

Why web design sucks now

How to add documentation to your product life cycle

The “throw it over the wall” model

Advantages

Disadvantages

The “put out the latest fire” model

Advantages