Direct answer
Cloudflare upgraded Markdown for Agents on October 9, 2026. HTML is now converted by an in-process streaming engine at the edge instead of being buffered and sent to a separate conversion service; the decompressed HTML limit increased from 2 MiB to 6 MiB; and converted responses no longer generate `x-markdown-tokens`, `x-original-tokens` or `Content-Length`. The larger limit brings more long product pages, knowledge bases and technical documents into conversion range, but it does not make longer pages better or guarantee that any AI system will request Markdown, crawl, index or cite the page. Treat HTML and Markdown as negotiated representations of the same URL and validate `Vary: Accept`, Content Signal, metadata, core facts, links, JSON-LD, caching and streaming clients separately.
Facts and background
Cloudflare published an update to Markdown for Agents on October 9, 2026. When an enabled Cloudflare zone receives a request that includes Accept: text/markdown, the edge can fetch the origin HTML and return a Markdown representation. The new implementation performs streaming conversion in process at the edge: it converts content as the HTML arrives instead of buffering the complete response and sending it to a separate conversion service. Cloudflare says this reduces conversion overhead and memory use.
Three interface boundaries changed with that architecture. The conversion limit increased from 2 MiB to 6 MiB, or 6,291,456 bytes, measured on decompressed HTML. Converted responses no longer generate x-markdown-tokens or x-original-tokens. Because the Markdown body is streamed, Cloudflare no longer recalculates or returns Content-Length. This is an update to representation and delivery, not a new search ranking, AI citation or indexing mechanism.
What changed, and what did not
| Area | Current behavior after October 9 | What it does not prove |
|---|---|---|
| Conversion architecture | In-process streaming conversion at the edge as HTML arrives | Every AI client will be faster or choose Markdown |
| Page limit | Up to 6 MiB of decompressed HTML, previously 2 MiB | Brands should expand pages until they approach 6 MiB |
| Token headers | No x-markdown-tokens or x-original-tokens | Markdown has no token cost or needs no context budget |
| Body length | No recalculated Content-Length for the streamed response | Clients can ignore timeouts, read limits or interrupted streams |
| Search and AI outcomes | A cleaner representation for clients that negotiate Markdown | Automatic crawling, indexing, citation, ranking or conversion |
The 6 MiB value is a conversion threshold, not a content-quality target. Pages above the old 2 MiB limit may now qualify, which matters for technical tables, product families, long documentation and help centers. But HTML inflated by navigation, embedded state, tracking scripts and repeated components still consumes origin, edge and client resources. Successful conversion only shows that the page stayed within the current size boundary; it does not show that the Markdown is complete, well prioritized or citable.
Treat HTML and Markdown as two representations of one URL
Markdown for Agents uses HTTP content negotiation rather than creating a separate .md URL for every page. The requester expresses a preference for text/markdown in the Accept header. Cloudflare fetches HTML from the origin, converts it and returns Content-Type: text/markdown; charset=utf-8. Current documentation says Vary includes Accept while preserving the origin's other Vary dimensions, allowing caches to separate HTML and Markdown variants.
| Request | Expected response | Common failure |
|---|---|---|
| Ordinary browser | Usable HTML with the correct canonical and full interaction | A rule mistakenly sends Markdown to human traffic |
Accept: text/markdown | Markdown media type, Vary: Accept and complete core facts | The cache key ignores representation and serves the wrong variant |
| AI bot requesting Markdown | Access policy, negotiation and use signal all match the intended policy | The team allows a User-Agent but never verifies the returned representation |
| AI bot accepting only HTML | Normal HTML; the feature must not force the client to switch | Support for Markdown is confused with universal bot adoption |
Converted responses preserve security, cross-origin and cache-relevant origin headers, but modify or remove fields that no longer describe the new body. Content-Type becomes Markdown, Vary includes Accept, and Content-Length is removed for streaming. Content-Encoding, Content-Range, Transfer-Encoding, ETag and Last-Modified are also removed. Clients that depend on these fields for conditional requests, completeness checks or caching need to follow the current contract instead of assuming the former headers remain.
Conversion restructures the page, so validate the facts again
Cloudflare currently returns three layers: YAML frontmatter extracted from meta tags, Markdown converted from the document body, and JSON-LD appended in a fenced code block. For titles, meta name="title" takes priority and og:title is only a fallback. Descriptions follow the same priority, while the image comes from og:image. Non-content elements such as headers, footers, navigation, scripts and styles are stripped during preprocessing. JSON-LD is the only script content preserved.
A visually correct HTML page is therefore not sufficient evidence. Compare the converted response page by page: does the H1 identify the same entity as the frontmatter title; is the description current; are model, specification, market, date and limitation facts complete; are tables readable; do internal links resolve correctly; and does JSON-LD agree with visible content? If core facts appear only after client-side JavaScript runs, the initial origin HTML may not contain them. Cloudflare defines Markdown for Agents as HTML conversion and points browser-rendered use cases to Browser Run's /markdown capability. Teams should not substitute “the final browser view looks right” for an actual negotiated-response check.
Decide the Content Signal before accepting the default
Current documentation says that if the origin already returns a content-signal header, Markdown for Agents preserves it and treats the origin policy as authoritative. If the origin does not return one, the converted response receives the default ai-train=yes, search=yes, ai-input=yes. These purposes cover AI training, search results and AI input, including agentic use.
Enabling conversion is therefore not only a format decision; pages that previously had no explicit use statement may receive a default set of affirmative signals. Global brands should review rights by content class. Owned public product and knowledge content, third-party licensed material, customer cases, paid content, distributor assets and user-generated content should not automatically share one policy. When content contains restricted or third-party rights, set the appropriate Content Signal at the origin and continue to protect nonpublic resources with authentication, authorization, network isolation and WAF controls. Content Signal expresses use preferences; it is not a confidentiality mechanism.
After token headers disappear, clients need their own budgets and failure classes
Cloudflare's original announcement showed x-markdown-tokens as an input for context sizing and chunking. Since October 9, Markdown for Agents responses no longer generate that header or x-original-tokens. Any collector that used these values for quotas, routing, chunking or cost estimates must migrate: enforce maximum bytes and timeouts while reading the stream, calculate tokens after completion or incrementally, and record the tokenizer and model context used.
| Failure class | Evidence to record | Do not misclassify it as |
|---|---|---|
| Negotiation failure | Request headers, status, Content-Type and Vary | The page is not indexed |
| Size or conversion failure | Decompressed HTML bytes, template, response body and error | An AI system rejected the brand |
| Interrupted stream | Bytes read, duration, connection state and retries | A complete Markdown response |
| Semantic loss | Missing titles, fields, tables, links, JSON-LD and source version | Success because the status was 200 |
| No external source appearance | Fixed question, platform, date, answer and source URL | Direct proof that Markdown configuration failed |
Use a repeatable acceptance protocol, not one curl request
Minimum acceptance covers both ordinary HTML and Markdown and samples the home page, product page, long knowledge article, table-heavy page, JavaScript-dependent page and a page near the size boundary. Preserve request headers, response headers, final URL, status, original HTML bytes, decompressed bytes, Markdown bytes, body hash and check time. Put title, direct answer, core parameters, limitations, sources, related links and JSON-LD in a field table and compare the two representations.
On October 12, Cloudflare also released the Agents SDK web_fetch tool, providing a concrete consumer for this acceptance model. The tool requests native Markdown first and can read the Markdown representation directly from sites that support Markdown for Agents. Its result includes the final URL, HTTP status, content type, total character count and continuation information, while long pages are returned in bounded windows with offsets. Operators can also set allowed hosts, response-byte limits, timeouts and redirect limits. This shows that Markdown negotiation is entering reusable agent-reading pipelines, but it also reinforces that “AI-readable” acceptance must record the final URL, status, media type, window completeness and request policy instead of relying on one successful response.
| Acceptance layer | Pass condition | What it cannot prove |
|---|---|---|
| Admission | Target clients can access the page and status and WAF policy are correct | A search or AI platform will crawl it |
| Negotiation | HTML and Markdown split correctly by Accept without cache contamination | Every crawler supports negotiation |
| Completeness | Facts, tables, links, frontmatter and JSON-LD remain useful and consistent | The underlying facts or page quality are correct |
| Rights | Content Signal matches rights, public scope and use policy | Malicious scraping or misuse is prevented |
| External outcome | A fixed question set records crawling, source presentation, accuracy and date | Markdown alone caused a citation |
After rollout, logs should distinguish clients that request text/markdown from those that still fetch HTML, as well as status, bytes and abnormal high-frequency or long reads for each representation. Cloudflare Radar or site logs can help observe traffic, but “accessed,” “converted,” “indexed,” “cited” and “produced a business outcome” remain separate measurements.
Business impact
The 6 MiB limit reduces the chance that large knowledge bases, complex product catalogs and long technical documents fail conversion merely because their HTML exceeded 2 MiB. In-process streaming also better accommodates longer output. The opportunity for global brands is not to publish more bloated pages, but to let one authority URL provide structured, fact-consistent representations to human browsers and AI clients that support negotiation, while reducing navigation and script noise and preserving metadata and JSON-LD. The change also exposes fragile integrations. Collectors that depend on token headers lose a budget input, downloaders that require `Content-Length` may mishandle streaming, caches that ignore `Vary: Accept` can mix variants, and pages without an origin Content Signal receive the current affirmative default. Content, CDN, rights, data and AI-measurement teams should validate the feature together rather than treating it as a single SEO or developer toggle.
Zhihe Growth assessment
Markdown for Agents is a representation-layer optimization, not a new GEO shortcut. It can reduce HTML noise for specific clients, but it cannot repair incorrect facts, weak evidence, confused canonicals, missing entity relationships or an inaccessible origin. The 6 MiB ceiling increases tolerance; it should not encourage page bloat. Strong pages still make their conclusion, conditions, fields, sources and next step coherent within a smaller information budget. Zhihe Growth recommends the sequence “policy first, enable second, validate both representations third.” Decide which pages permit search, AI input and training; roll out by hostname or path; compare HTML and Markdown with a fixed field table; then observe actual logs and a fixed AI question set. Technical admission is complete only when content integrity, rights signals, cache separation and client reading all pass. Citation rate and business growth require separate longitudinal measurement.
Recommended actions
- Inventory Cloudflare root, language, documentation, store and SaaS custom hostnames, and choose the paths appropriate for Markdown for Agents.
- Measure compressed transfer size and decompressed HTML size for representative pages; flag pages above 2 MiB, near 6 MiB and above 6 MiB separately.
- Request each sample normally and with `Accept: text/markdown`, preserving status, `Content-Type`, `Vary`, Content Signal and the complete body.
- Confirm that the CDN, reverse proxies and downstream caches separate HTML and Markdown by `Accept` and never contaminate human and machine variants.
- Make `x-markdown-tokens`, `x-original-tokens` and `Content-Length` optional; implement streaming reads, timeouts, maximum bytes and local token calculation.
- Compare titles, descriptions, H1, direct answers, product fields, limitations, tables, links and JSON-LD across HTML and Markdown and record every difference.
- Define `ai-train`, `search` and `ai-input` policy by content class and set the required origin `content-signal` rather than accepting the affirmative default without review.
- Check whether initial HTML contains core facts for client-rendered pages; if not, improve server-visible content first or separately evaluate a browser-rendered conversion path.
- Segment HTML and Markdown clients, status, bytes, latency, failures and retries in logs; do not treat conversion success as indexing or citation.
- Use the same branded and non-branded question set to track source URLs, factual accuracy, citation context and business behavior, with non-enabled paths or pages as directional controls.
Limitations
This article confirms Cloudflare's public October 9, 2026 update to the Markdown for Agents architecture, 6 MiB decompressed HTML limit and response headers, plus the current documentation for negotiation, output structure, Content Signal and availability. Cloudflare still labels Markdown for Agents as Beta, so interfaces, limits, plans and defaults may change. Markdown for Agents converts HTML only. It does not automatically execute all client-side JavaScript or replace specialized handling for PDFs, images, Office files or authenticated content. The 6 MiB boundary applies to decompressed HTML, not compressed transfer size, and this article does not reinterpret it as a page-quality, search-crawl or model-context limit. An origin Content Signal is preserved, but Content Signal is not authentication, authorization, access control or legal advice. The default `ai-train=yes, search=yes, ai-input=yes` does not prove that a brand owns every asset or that every third party will honor the signal. Markdown, HTTP 200 and valid JSON-LD do not guarantee crawling, indexing, ranking, AI citation, traffic or conversion.