Why documentation won't go fully headless

I was at an event for Write the Docs a few weeks ago and someone asked me if I thought we would see fully headless documentation at some point.

I instinctively and reactively said “no” immediately, but I’ve been thinking about that question and my answer since then.

Headless documentation doesn’t refer to content produced by a headless horseman, but rather content that is not visible on a website by humans and instead available purely through programmatic or agent-accessible means.

Illustration of a computer terminal with a black background and green text of _docs

Currently, agents are accessing documentation through the web, crawling websites. If people stop visiting documentation sites, what other ways might we use to deliver documentation content to them where they are? Is there a potential future where documentation is published “headlessly”?

Agents are the primary audience of software documentation #

According to several documentation platforms (with ample aggregate server traffic data), agents outnumber human readers of documentation:

GitBook published Research: AI agents are now the majority reader of your docs using data from April and May:

When we look at purposeful reads — humans actively reading docs, and AI agents actively pulling them — AI has crossed 50% for the first time.

Mintlify, in their 2026 State of Knowledge Report on data from August:

2026 marks the first year on record that AI agent readership of Mintlify-powered sites crossed the majority line.

August alone saw 257 million agent web requests and 131 million human page loads. This means agents now account for over half of total traffic.

Fern published State of Docs: Agents are your top readers in November, saying:

Agents now make up more than half of the requests to the docs we host

With agents dominating requests to web servers hosting documentation, other interfaces are also gaining popularity as mechanisms to publish content to the audience of agents.

What is happening today? #

The internet offers many ways of accessing information, and other ways to publish information besides visibly as files accessible as HTML or Markdown on the web.

Beyond documentation websites, people are publishing content to agents in the following ways:

These mechanisms are all being used in addition to documentation content published on websites, including GitHub repositories for skill files. For a future with headless documentation, these mechanisms would be the only way of providing content to your users.

Will documentation sites become fully headless? I don’t think so.

Why headless documentation is not the future #

I don’t think that a fully headless documentation site is a viable future for technical content for most products, and not anytime soon. There’s several reasons why I think that:

Current agent tooling #

Agents are the majority of traffic to documentation web servers, but as far as I can tell, AI agents largely reuse human ways of interacting with information:

So far as I can tell, current agent tooling is built to reuse existing interaction patterns, rather than demand and expect agent-specific tooling to be built. While it’s true that many organizations are building and providing purpose-built tooling to support agentic interactions, the purpose is to support agent use, but there isn’t much about the tooling that makes it agent-specific.

The existing purpose-built tooling for agents (skills and MCP servers) are wrappers around pre-existing interaction methods used by humans — documentation and APIs. Other tooling exists, like “computer use”, to quite literally use interfaces designed and built for humans first.

Other tooling to support agentic use, like command-line interface (CLI) tools or software development kits (SDKs) serve to formalize and structure pre-existing interactions with APIs, thus providing a more structured interface for agents to use, but humans can easily use those tools too.

So long as AI agents succeed at prompted tasks using the same tools as humans, it’s unlikely that we’ll see tooling built exclusively for agents. Given the current state of agent tooling, providing documentation content without the web would mean ignoring interfaces that are actively used by AI agents. Of course, that’s not the only issue with providing headless-only documentation.

Content discovery #

The challenges of discovering content also make it unlikely that headless-only documentation is a practical offering. Why?

The web is passive. Headless content mechanisms are active.

An agent needs to find an SSH endpoint to use it. A user needs to install an MCP server or a CLI tool for the agent to explore the available options and use them. Before an agent can use a skill, the file needs to be created and added to a specific directory or referenced explicitly.

Some conventions exist to aid in discoverability, but they rely on the web:

Websites offer search-based discovery through a sitemap and/or an llms.txt file as a map for agents. Removing the site removes the discovery layer.

Providing an MCP server for agents to use might not be relevant in the future, and instead web applications and websites get more powerful for agents. An emerging standard, WebMCP allows an agent to invoke MCP-like tools directly from a web application — no backend MCP server required:

Unlike backend MCP servers that bypass the browser, WebMCP executes in the user’s tab, keeping active application state and visual UI in perfect sync.

With WebMCP, a company doesn’t need to set up an MCP server endpoint, and there’s nothing for a user to install and configure with their AI assistant.

Before AI agents even existed, large language models (LLMs) were trained on data scraped from the web. If your documentation content isn’t on the web (and able to be scraped), it’s unlikely that models will be trained on it and have even less of a chance to discover content about your product.

If you remove the head from content, agents are lost and require a lot more guidance to work. You also lose something when providing the content itself.

Auditable authoritative content #

Documentation is, for most software companies, the site where you publish the contractually relevant content that defines how your software works in supported ways. If the content is inaccurate or misleading and causes a customer to lose money or data due to a misconfiguration, you’re potentially exposed to some legal liability.

If your readers never interact directly with your authoritative source of truth, and instead only access content mediated by probabilistic tools, that likely exposes your company to some legal liability. I’m not a lawyer, but I wouldn’t want to stake the trust of all of my customers and the future of my company on the reliability and accuracy of an AI agent.

If your documentation website serves as an auditable, authoritative source for how your product works, you can’t cut off its head.

Observability and evaluation #

Because headless content lacks a website, the AI interface is what your readers interact with instead. If an AI agent is responsible for providing the content to your readers, the agent effectively owns the correctness of your content — so you need a way to validate it. Traces and logs provide the necessary observability so that you can review and validate that the content being generated for your users is accurate and correctly reflects the supported capabilities of your product.

AI is a practical tool for solving computationally verifiable problems. If you can computationally verify the output of the system, you can This is a main reason why there are so many data analysis agents — the agents can generate SQL, which can be tested relatively reliably to see if it both runs and returns the expected result.

However, the correctness and accuracy of language are not computationally verifiable and require a large amount of (usually human) review and evaluation to (usually manually) validate. If the testing improves for language accuracy, it might be possible to provide headless-only documentation, but the costs seem quite high.

Differing audience needs #

If those aren’t enough reasons to make the case that headless-only documentation isn’t practical, there’s also the perennial consideration — your audience.

It’s telling, I think, that I was asked about headless documentation at a Write the Docs event for San Francisco Tech Week. It’s indicative of the AI-obsessed bubble encircling the Bay Area at the moment.

Not every audience of every product that provides documentation is actively using AI tools, or is interested in using AI tools, to access content about how the product works. Your audience might be in a heavily regulated industry. Your product might be used in areas with slow Internet connections and by users on devices that don’t have a lot of memory. An AI-first headless documentation strategy is a poor fit for those audiences.

Build for all audiences instead #

Headless-only documentation isn’t the future, but if your audience uses AI agents to learn about your product, it’s worthwhile to spend time on providing content that meets your readers where they are.

How do we strategically provide documentation in an AI-first web? Provide content for all your audiences (those with heads and the machine-based users that do not), and make sure your content is high quality.

AI tool adoption for some users has meant different interaction patterns for your content, so it’s worth it to explore different ways to support those interactions. However, the mechanism is not quite as important as the reliability and usability of the content.

As GitBook puts it in their research, accessing the content is only part of the problem. The more important consideration is “whether those agents can find and use your docs reliably, not just request them.”

Maintain high-quality, authoritative content in your documentation, provide mechanisms for agents to reliably use that content, and evaluate agent performance if possible.

If you’re not sure where to start, check out Developing an AI strategy for documentation