Long documents need retrievable sections, but shorter is not automatically better. “220 V” has no object. “Can be used” has no scope. “20” in an isolated table cell has no unit. Design chunks around evidence completeness rather than a universal paragraph-length rule.
What fixed windows can lose
Synthetic record D1 describes AX-220 power, capacity and usage restrictions. The program splits the same text into 32-, 64- and 128-character windows and retains every window. It also supplies author-structured comparison excerpts. These are explicitly manual examples, not a claim that the code automatically understands manuals.
AX-220 | electrical | Supply 220 V only. | D1 | v1
AX-220 | use | Indoor sealed hard floors. Do not use outdoors or on untreated wood. | D1 | v1
AX-220 | tank | Tank 20 L. | D1 | v1
Both representations appear in the chunking field of the execution output. The program checks whether a single chunk retains both the model identifier and each complete, author-selected constraint phrase. The actual synthetic example produces:
| Method | Chunks | Voltage and model together | Capacity and model together | Full usage limits and model together |
|---|---|---|---|---|
| 32-character window | 6 | No | No | No |
| 64-character window | 3 | Yes | Yes | No |
| 128-character window | 2 | Yes | Yes | Yes |
| Manually structured excerpts | 3 | Yes | Yes | Yes |
These are literal preservation checks, not semantic entailment judgments or retrieval and generation scores. The 128-character window also preserves all three conditions in this short example, so the result does not establish universal superiority of structured chunking. Character counts are neither token counts nor recommended web paragraph lengths; the relationships vary across languages and tokenizers.
Preserve heading and table relationships
A parameter row should retain its product ID, column heading, value, unit, footnotes and version. Removing “charger input” can make 220 V appear to be motor voltage. Retaining only “hard floors” can discard “sealed” and “indoor.”
| Structure | Retain with the excerpt | Do not infer |
|---|---|---|
| Manual section | Version, heading path and location | Current version from filename alone |
| Parameter table | Row object, column, unit and footnote | A blank cell inherits the previous row |
| Restriction | Subject, negation and exceptions | A condition without its object |
| Source relation | Original document ID and excerpt range | A generic homepage is sufficient provenance |
Verify parsing before optimizing chunking. If extraction already shifted a table column, embeddings cannot reliably restore the original meaning. A malformed electrical or usage restriction should block admission to a trusted knowledge collection.
Context enrichment has its own risks
Anthropic's Contextual Retrieval description explores adding document context to excerpts. It is a possible internal retrieval technique, not a ranking requirement for every public website. Separate source text from generated additions and record both provenance and transformation.
An added explanation may confuse a product family with an exact model, import an obsolete specification or rewrite “unconfirmed” as “supported.” Compare enriched fields against the source. Repeating an unsupported explanation does not turn it into a fact.
Design a fair chunking comparison
Hold the document, questions, encoder and relevance key fixed while changing segmentation. Record chunk counts and sizes, top-k retrieval, condition preservation and final claim support. When chunk IDs change, map labels to original evidence locations rather than reusing incompatible IDs.
The current package executes three character windows, a manual structured comparison and literal constraint checks. It does not run automatic PDF parsing or a complete retrieval and generated-answer benchmark over multiple chunk sizes. No percentage improvement is claimed. These remain separate follow-up experiments; a constraint check is not statistical evidence of improved answers.
Public articles do not need a universal chunk size
Google's AI optimization guidance does not require special website chunking rules. A team controls its own RAG windows but does not fully control how an external service segments a page. Clear headings, definitions, tables, restrictions and sources are useful without mechanically equalizing paragraph lengths.
For Chinese B2B exporters, Zhihe Growth's editorial task is to preserve equivalent facts and limitations across Chinese and English materials. That reduces ambiguity in the source itself. Whether an external platform cites the page still requires a separate real-world test.
Use an explicit chunk acceptance sheet
Mark critical source constraints before choosing a window. The AX-220 example requires its identifier, 220 V, 20 L, indoor use, sealed hard floors and both prohibited uses. Replacing “indoor sealed hard floors” with “hard floors” expands scope and should fail, even though a keyword remains. Copy a table row without its header into a blank document: if the numbers no longer have a clear meaning, restore the header relationship.
Keep three columns: source location, retained content and the error an omission could cause. Repeat the sheet after changing window or heading rules. Count retrievable chunks separately from independent facts: three overlapping copies of one sentence are not three sources.
A future design could index a short child passage and retain a locator to its parent section for expansion. This package does not implement that system. Expansion still needs a context budget and must not import another model's parameters from the same chapter. Include negative questions such as unsuitable surfaces, not only product benefits. A method that preserves selling points but loses prohibitions is not ready to support procurement advice. Automatic parsing and scaled acceptance checks require their own validation.
Use the complete teaching package with the original dataset. Continue with independently verifiable fact units, provenance and citation support checks. Short FAQs provide direct answers; long-form guides explain mechanisms and failure boundaries.