Docs as code is a much-vaunted workflow and toolchain for writing, publishing, and maintaining technical documentation — but in practice, docs as code doesn’t deliver on its promise.

What is docs as code?

According to the Docs as Code page in the Documentation guide for Write the Docs:

Documentation as Code (Docs as Code) refers to a philosophy that you should be writing documentation with the same tools as code:

• Issue Trackers
• Version Control (Git)
• Plain Text Markup (Markdown, reStructuredText, Asciidoc)
• Code Reviews
• Automated Tests

This means following the same workflows as development teams, and being integrated in the product team. It enables a culture where writers and developers both feel ownership of documentation, and work together to make it as good as possible.

This is promising — shared ownership of documentation and a shared goal to make documentation as good as possible by sharing the same tools and processes.

All too often, however, I find that this philosophy is adopted as a set of tools, and the processes and integration are ignored. Even if you have processes in place, the tools you use to do docs as code make it easy to circumvent the processes, even unintentionally.

The promise of docs as code

If your documentation workflows follow the development workflows, as a technical writer, you can get your job done more easily.

If you do docs as code…

• Documentation reviews can take the same format as code reviews—a pull request—making it easier to get reviews from engineers.
• Code samples can be automatically tested, or better yet, pulled in at build time from the product code.
• Doc changes can happen in sync with the product changes if docs are in the same repo, streamlining the release cycle.
• Documentation can be versioned, and you can keep track of when and how a given page was last updated.

The pitfalls of docs as code

Writing content isn’t the same as developing code, and as a result, doing docs as code has pitfalls:

• Git is confusing
• Processes must be defined and reflected in Git workflows
• Tools to write docs can be inconsistent
• Merge gates and build checks are great… if you have them
• Reviewing content in the repository is hard to read

Let me explain…

Git is confusing

To do docs as code, writers need to learn how to use and troubleshoot Git. And Git isn’t simple — Julia Evans made an entire Zine about it and has been working on a longer series about Git concepts to make it much clearer.

There is a deep level of complexity with Git, and while you can go a long way only knowing how to git fetch, git pull, and git merge, it’s easy to get into an unexpected state with your cloned Git repo, local branch, or disastrous merge conflict resolution decisions. In those cases, often the best troubleshooting is starting over — and that’s not a great experience!

Processes must be defined and reflected in Git workflows

When you use a docs as code workflow, you need to codify your docs processes and instantiate them in your Git workflow. So you not only need to define the following:

• When to publish doc updates
• How to release doc updates
• How to coordinate a docs release with a product release

You also need to define Git best practices for your team about how to manage those, such as:

• Whether to use release branches, or merge pull requests frequently but publishing infrequently.
• Whether to use Git rebase or Git merge to maintain Git history in a given branch.
• Whether and how to use feature branches and pull requests.
• Whether to squash merge pull requests to main.

Even if you manage to define best practices that your team is committed to following, there isn’t a way to force your documentation contributors to adhere to all of these best practices. Due to the lack of enforcement of these best practices, you can easily end up in a situation where writers follow slightly different practices based on what their tools make easy to do.

As just one example, I spent several weeks merging all my pull requests to main, instead of squash merging as was the best practice. At some point, I’d needed to do a regular merge and preserve all my branch commits on main, and GitHub “helpfully” remembered my last-used setting.

Because the best practice was just a practice, and not enforced, it took me several weeks of cluttering up main with my chaotic branch commit history before I realized what was happening. In the meantime, the best practice of squash merging, meant to maintain a clear association between a PR of feature changes and a Jira ticket, was completely ignored with no warning.

Tools to write docs can be inconsistent

In a typical docs as code environment, you write locally and push your changes up to a central repository. Unlike a typical content management system (CMS) like WordPress or Drupal, or software like MadCap Flare, the writing environment isn’t shared — only the content is the same.

Instead, the writing environment is relatively uncontrolled by default — a writer can choose to write their content in whatever system they want, so long as they can commit text to the Git repository.

A writer contributing to the documentation can write in a basic text editor like Sublime Text, an extension-filled Visual Studio Code setup, in vim, or some other tool — even the GitHub or GitLab UI!

Because only the end result matters to docs as code, different writers and documentation contributors can use different tools with different settings and functionality. However, these environment inconsistencies can lead to inconsistent content quality and style, and make issues with content development more difficult to troubleshoot.

If you use a text editor that doesn’t have a spell checker installed, you can easily introduce typos into the documentation — but only the topics that you contribute to. If your colleague doesn’t use a system supported by the automated style checker recommended by your team, the style of their documentation topics will deviate from company style.

Merge gates and build checks are great… if you have them

One advantage of docs as code is the extensibility and automated nature of the workflows. Because you can treat documentation content like code, you can implement some code-like practices, like checking the validity of your documentation markup when you propose changes to the documentation with your pull request (a merge gate) or when you build the documentation site (a build check).

However, to take advantage of these capabilities, you often need to build the tools and checks yourself, or get your documentation platform team to build them for you — right after they work on the rest of the backlog of improvements for the customer-facing documentation site.

Some companies put a lot of effort into engineering efficiency, with entire developer experience teams devoted to improving engineering workflows. Meanwhile, documentation teams are often lucky to get one engineer to work on the documentation site itself, let alone help enable any special docs as code workflows. This makes sense for the business, but it makes it difficult to take advantage of what docs as code can do for you.

For example, you could set up merge gates or build checks to perform documentation quality improvement tasks like the following:

• Optimize image size
• Lint content to standardize and correct markup
• Check for broken links
• Test code examples
• Validate adherence to the style guide
• Check spelling and grammar

Some of these might make more sense to implement in your writing tool directly (such as with Visual Studio Code extensions), but then you would need to require writers to all use the same writing tools configured in the same way.

Without resources to implement custom merge gates or build checks for documentation quality, documentation writers have the added process burden of opening a pull request, requesting reviews and approvals, and miss out on the potential advantage of automating tedious tasks. And if their content is in the same repository as the product code, they also need to wait for a bunch of unrelated tests to pass before they can merge any doc updates.

Reviewing content in the repository is hard to read

A much-touted benefit to docs as code is that the content is in the same tools used by engineers, which makes it easy to get technical reviews done in GitHub, GitLab, or your Git provider of choice.

Unfortunately, documentation reviews in a pull request can be confusing. Just like reviewing UI code is difficult if you only have access to the source and not a staging environment, if you can’t provide your reviewers with a staged or preview version of the content, you might get a technical review full of confused comments instead of helpful feedback.

An engineer might see that you deleted lines from a file, not realizing that you deleted them because they were in the wrong place, not because the information was wrong.

Someone else might comment that you need to provide a note on several pages, not realizing that you did that — using a single sourced content reference that isn’t obvious to someone who doesn’t write documentation.

Parsing a documentation pull request and providing helpful comments requires the reviewer experience the content the same way that a future customer will — but a pull request only provides the raw, marked up text.

Upholding the docs as code promise

In its current state, with these pitfalls I’ve outlined (and more), I find it problematic to recommend docs as code as a one-size-fits-all solution for all your technical writing CMS needs. Docs as code is a workflow with processes and tools, and therefore requires investment, maintenance, and a decent amount of custom tooling to get true value out of it.

It’s possible that using a well-resourced static site generator like Docusaurus or MkDocs as part of your docs as code workflow might address some of these pitfalls, but from my impressions writing, building, and deploying this blog using the Hugo static site generator, it’s unlikely. I’d guess that those give you extensions and flexibility to make a static site work well for documentation, but they don’t solve the core writing experience challenges of a docs as code workflow.

If you’re considering implementing docs as code workflows, you can choose the degree to which you want to adopt the practices. If you require a branch, pull request, and all build checks to pass to fix a typo, the time it takes to fix a typo could easily triple. On the other hand, with that level of overhead, you gain ultimate auditability of documentation changes — the what, when, why, and by whom.

When considering docs as code workflows today, I often see what my friend referred to as the “most extreme option” — a writing team and toolchain that has adopted all the practices from code development. I think to uphold the promise of docs as code, we could do to add a bit more of the docs back into the workflow.

As Fabrizio Ferri Benedetti put it in his post, The pros and cons of using Markdown ¹:

you want the docs to be the product […] But be very careful about the product not being, for example, the pipelines or the site you’re gonna render.

There are newer CMSes on the market, like ReadMe, Heretto, Paligo, and others. From my extremely limited experience using one of these CMSes, it seems like the writing experience is much simpler, and there is a way to write Markdown without needing to use developer level tools.

Unfortunately, it seems like these tools also still need to mature and borrow more of the practices that do work in docs as code, like being able to compartmentalize documentation drafts and release specific changes at a specific time.

It could be as well that we are early enough in the docs as code ecosystem that the tooling hasn’t yet matured—but Anne Gentle’s book Docs Like Code came out in 2017, so this concept has been around for some time.

I sincerely hope that the content management systems improve, or that we see more standardized toolchains with guardrails that enforce best practices for doing docs as code.

The easier it is to implement docs as code — consistently, without much room for individual writer error — the easier it is to focus on producing high quality documentation, together. This is exactly why large companies have entire engineering efficiency teams.

What would it take to get a writer efficiency community to focus on a spectacular docs as code experience?