…
```
An agent fetching that page receives Tailwind utility classes, responsive grid wrappers, and raw JavaScript functions. It has to reconstruct the actual document content from the props of an unrendered component.
The [Fumadocs](https://www.fumadocs.dev/) framework documents this behaviour as its intentional default.
While not as bad as serving raw HTML with no alternative and forcing the LLM to parse through a messy DOM tree, it nevertheless exposes an interesting gap in currently available tooling.
By contrast, fetching Vercel's documentation returns a clean markdown document:
```md
---
title: Vercel Functions
product: vercel
url: /docs/functions
canonical_url: "https://vercel.com/docs/functions"
last_updated: 2026-07-15
type: conceptual
…
---
# Vercel Functions
When you deploy your application, Vercel automatically sets up
the tools and optimizations for your chosen [framework](/docs/frameworks).
It ensures low latency by routing traffic through Vercel's [CDN](/docs/cdn),
and placing your functions in a specific region when you need more control
over [data locality](/docs/functions#functions-and-your-data-source).
```
The takeaway from this should be obvious. Serving agent-ready content requires compiling the source into distinct representations: HTML for humans and markdown for agents.
While this post focuses on `.mdx` sources specifically, the approaches outlined here apply to any structured documentation format.
## HTML converters versus raw MDX
Serving markdown from docs-as-code repositories generally follows one of several strategies:
### Converting HTML
Some platforms run an edge HTML-to-markdown converter (like [Turndown](https://github.com/mixmark-io/turndown) or Mozilla Readability) over the generated HTML page. Cloudflare, for example, offers this at the CDN layer.
While HTML conversion requires zero per-component maintenance, it destroys structured content.
Syntax-highlighted code blocks are the clearest casualty. Modern frameworks wrap highlighted code tokens in nested `
` elements and custom line containers (for per-line highlighting, etc.), and converting that markup back to text frequently corrupts indentation, tabs, and language identifiers. Interactive tabs, accordion groups, and multi-column grids collapse into generic `` hierarchies that lose their contextual relationships.
### Raw source
Serving the raw `.mdx` file directly is the cheapest option, but it offloads compilation artifacts onto the reader:
- Unrendered JSX tags (`
`, ``) clutter the text.
- Build-time dynamic helpers (such as file snippet embedders or automatic cross-reference resolvers) remain unresolved.
- Unused client imports and exported configuration objects waste model context and tokens.
### AST reparse
To generate a stable, most faithful representation, you fork at the MDX source parse. First, generate an abstract syntax tree, transform custom JSX nodes into their semantic markdown equivalents, then stringify to clean markdown. The UnifiedJS ecosystem is very well suited for JSX/MDX based sources.
## The pipeline
_The page pipeline generates HTML; the projection pipeline emits clean markdown._
Reusing the framework's standard MDX compiler (such as `@next/mdx`) for the markdown projection does not work. Framework compilers compile `.mdx` files into executable JavaScript modules, running rehype plugins (such as `rehype-pretty-code`) that transform markdown syntax into React JSX trees and styled HTML elements.
To retain semantic markdown nodes, the projection pipeline parses the raw file string independently using `remark-parse` and `remark-mdx`, both from the [UnifiedJS](https://unifiedjs.com/) ecosystem:
```ts
/**
* Compiles an MDX source document into self-contained, clean markdown.
*
* Prunes authoring frontmatter, projects custom JSX elements into standard mdast
* structures, normalises paragraph blocks, and absolutises site URLs.
*
* @param source - Raw MDX source content including frontmatter.
* @param origin - Base site origin used to resolve relative links.
* @param options - Projection and diagnostic configuration.
* @returns Serialised markdown string.
* @throws {ProjectionError} When encountering unregistered tags, dynamic expressions, or invalid block nesting.
*/
export function projectMdxToMarkdown(
source: string,
origin: string,
options: ProjectOptions = {},
): string {
// Creates mdast from the source MDX file.
const tree = unified()
.use(remarkParse)
.use(remarkGfm)
.use(remarkFrontmatter, ["yaml"])
.use(remarkMdx)
.parse(source) as Root;
reduceFrontmatter(tree, options.frontmatter);
try {
// Applies per-tag projections.
tree.children = projectChildren(tree.children) as Root["children"];
liftBlocksFromParagraphs(tree);
} catch (error) {
// Surfaces line:column coordinates for authoring errors.
if (error instanceof ProjectionError && options.file) {
throw atFile(options.file, error);
}
throw error;
}
rewriteUrls(tree, origin);
// Turns the augmented mdast into a markdown document.
return unified()
.use(remarkStringify, {
bullet: "-",
emphasis: "_",
fences: true,
listItemIndent: "one",
rule: "-",
})
.use(remarkGfm)
.use(remarkFrontmatter, ["yaml"])
.stringify(tree);
}
```
## Component projections
A component projection is a pure function that transforms a JSX AST node into standard mdast nodes:
```ts
/**
* Result of projecting a JSX element: transformed mdast node(s), or null to drop.
*/
export type ProjectionResult = RootContent | RootContent[] | null;
/**
* Transforms an MDX JSX element into standard mdast markdown nodes.
*
* @param node - The JSX element node with its attributes and unprojected children.
* @param projectChildren - Recursive projection function for child node lists.
* @returns Transformed mdast node(s), or null to drop the element.
*/
export type Projection = (
node: MdxJsxElement,
projectChildren: (nodes: RootContent[]) => RootContent[],
) => ProjectionResult;
```
Binding a projection to every tag declared across your site turns "what does this component mean in markdown?" into a concrete type contract. Answering that question is where the actual work lives, and the implementation depends entirely on what the original component was designed to do.
### Unwrap and drop
Tags that exist purely to colour, wrap, or position their children carry no structural meaning for an agent. In those cases, the projection returns the projected children directly and the outer wrapper drops away:
```mdx
Status: ok today.
```
```md
Status: ok today.
```
Column sets, responsive grids, and card decks follow the exact same pattern. Elements that exist purely for human interactivity such as code block copy buttons resolve to nothing. Audience-conditional blocks resolve according to which branch is allowed to survive:
```mdx
Only people read this.
Only agents read this.
```
```md
Only agents read this.
```
Both can be defined as shared helpers in your tag registry.
```ts
/** Drops the element from markdown output entirely. */
export const dropFromMarkdown: Projection = async () => null;
/** Unwraps children inline. */
export const unwrapInMarkdown: Projection = async (node) =>
node.children as RootContent[];
```
### Structural tags
Markdown already provides native constructs for the components used in most documentation styleguides. A callout banner maps naturally to a blockquote prefixed with a bold label:
```mdx
Mind the gap.
Mind the gap.
```
```md
> **Note: Careful**
>
> Mind the gap.
> **Note:** Mind the gap.
```
When a tag can appear in both block and inline contexts, its projection should adapt accordingly. An inline tag sitting within a sentence produces an `inlineCode` node, whereas that same tag standing on its own line receives blockquote formatting. Use the AST node's own type (`mdxJsxTextElement` vs. `mdxJsxFlowElement`) to choose the right output.
### Hidden content
Disclosure widgets, tab sets, and accordions show only one piece of content at a time in HTML. Markdown has no client-side state or click handlers to toggle visibility; preserving that hidden state in a projection simply throws content away that a human reader can readily reach.
Instead, the projection expands every branch beneath a descriptive heading:
```mdx
Some _body_ text, then a list:
- one item
- another
```
```md
**Build output**
Some _body_ text, then a list:
- one item
- another
```
Tab sets follow the same principle: the container unwraps its children back through the AST walker, emitting each tab's contents beneath its corresponding label.
### Build helpers
Unresolved dynamic helpers are one of the primary flaws of serving raw MDX to LLMs. Because the projection pipeline runs during the static build with direct access to the filesystem and site metadata, it can resolve dynamic helpers into literal content.
Every code plate in this post uses a `` tag: it references a source file path and an optional region ID, and the projection reads the file off disk to emit a syntax-highlighted code fence. Similarly, an internal `` resolves the target document's title and appends a `.md` suffix to the URL, converting an empty tag into an actionable link:
```mdx
```
```md
[Customizing the Claude Code statusline](https://example.com/blog/cc-statusline.md)
```
Because projections operate as build-time AST transforms rather than simple string replacements, broken file paths or dead internal links can fail the static build immediately before reaching production.
## Block lifting
In MDX, authors frequently place custom component tags inline within regular paragraphs:
```mdx
Check out our utility configuration: for origin handling.
```
When `` transforms into a fenced code block, having a block-level node nested inside a `paragraph` AST node produces invalid parse. Standard serialisers like `remark-stringify` will emit the code block without the mandatory surrounding blank lines, leading to corrupted outputs.
To keep the AST valid, a lifting pass traverses the tree after component transformation and splits parent paragraphs around block nodes:
_The lifting pass splits parent paragraphs around block nodes to preserve mdast hierarchy._
The lifting pass ensures every block-level node sits at the top level of the AST:
```ts
/**
* Recursively extracts block-level nodes out of enclosing paragraph nodes.
*
* Splits parent paragraphs around block nodes so serializers emit proper
* blank line separators around fences, blockquotes, and lists.
*
* @param parent - AST parent node whose subtree is rewritten in place.
* @throws {ProjectionError} When a block node resides within an unliftable phrasing container such as a table cell.
*/
export function liftBlocksFromParagraphs(parent: Parents): void {
if (parent.type === "tableCell") {
for (const child of parent.children as RootContent[]) {
if (BLOCK_TYPES.has(child.type) && child.type !== "paragraph") {
throw new ProjectionError(
`markdown-projection: a <${child.type}> block cannot be projected inside a table cell.`,
parent.position,
);
}
}
}
const out: RootContent[] = [];
for (const child of parent.children as RootContent[]) {
if (child.type !== "paragraph") {
if ("children" in child) liftBlocksFromParagraphs(child as Parents);
out.push(child);
continue;
}
/**
* Emits a phrasing run as a paragraph node, discarding empty whitespace runs.
*
* @param group - Accumulated phrasing nodes.
*/
const pushPhrasing = (group: PhrasingContent[]): void => {
const trimmed = trimPhrasing(group);
if (trimmed) {
out.push({ type: "paragraph", children: trimmed } satisfies Paragraph);
}
};
const phrasing: PhrasingContent[] = [];
for (const sub of child.children) {
if (!BLOCK_TYPES.has(sub.type)) {
phrasing.push(sub);
continue;
}
pushPhrasing(phrasing.splice(0));
if ("children" in sub) liftBlocksFromParagraphs(sub as Parents);
out.push(sub as RootContent);
}
pushPhrasing(phrasing);
}
(parent as { children: RootContent[] }).children = out;
}
```
The inverse can occur as well. A tag written alone on its own line occupies a block slot in the MDX AST, but its projection might produce inline phrasing content. For instance, `` and `` each return a single `link` node.
```ts
/**
* Projects a downloadable file reference into a markdown hyperlink.
*
* @param node - The Download JSX element AST node.
* @returns Link AST node pointing to the target asset URL.
* @throws {Error} When `src` attribute is missing or non-literal.
*/
Download: (node) => {
const src = getAttr(node, "src");
if (!src) {
throw new Error(
"markdown-projection: requires a literal src attribute.",
);
}
// Returns a mdast link node.
return {
type: "link",
url: src,
children: [
{ type: "text", value: getAttr(node, "label") ?? publicFileName(src) },
],
};
},
```
If emitted into a block slot without a parent paragraph, the converter prepends to adjacent paragraphs without any separation or whitespace.
```ts
// Skips block elements.
if (node.type !== "mdxJsxFlowElement" || result == null) return result;
// Ensures a block element for orphaned inline (phrasing) elements.
return groupIntoBlocks(Array.isArray(result) ? result : [result]);
```
---
Sources:
- UnifiedJS —
- IMG.LY, "Make Your Docs Agent-Ready: MDX to Markdown" —
- Fumadocs, "Remark LLMs" and "Markdown Rendering" —
- Cloudflare, "Markdown for Agents" —
- mdast syntax —
…