# GEO and AI search engineering teaching labs

Zhihe Growth. Synthetic teaching material, not client data or a commercial AI-platform retest.

## Four components

1. Retrieval: SQLite FTS5 BM25, actual embeddings, RRF and an actual cross-encoder. All four per-query rankings and dense and cross-encoder scores are retained. Raw BM25 scores are not separately saved; RRF contributions can be recomputed from the recorded candidate ranks and constant 60.
2. Facts and sources: JSON Schema validation and four author-labeled support examples, not automated semantic truth verification.
3. Measurement: Wilson and paired-question bootstrap examples, not evidence of commercial improvement.
4. Bilingual consistency: identity, numeric, scope and version fields, not automatic natural-language translation assessment.

Chunking examples compare 32-, 64- and 128-character windows with manually structured excerpts. Checks require a model identifier and complete constraint phrases in one chunk. These are literal checks, not semantic truth or answer scores. There is no PDF parser or generation-model call.

## Contents and reproduction

`data.json` contains nine synthetic English records, ten authored queries, grades, bilingual facts and repeated synthetic observations. `labs.py` executes on CPU. `fact.schema.json` defines the complete schema. `model-revisions.json` pins the two model revisions. `results.json` records actual execution, UTC time, input hashes, environment, rankings, scores, timings and limitations. `lab-dependencies.txt` lists direct dependencies; `requirements-lock.txt` records the complete observed environment and may need platform-specific resolution. `SHA256SUMS.txt` checks integrity, not factual truth or model quality.

Run in the extracted directory:

```sh
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r lab-dependencies.txt
python labs.py --output my-results.json
```

Initial model downloads require internet access, bandwidth and local disk space. Weights are not redistributed. Model cards and licenses remain at the original model repositories; these are not proprietary Zhihe models.

Once the pinned weights are cached:

```sh
python labs.py --offline --output my-offline-results.json
```

Keep `model-revisions.json`. Deleting it causes an online run to resolve current revisions instead of reproducing the pinned configuration. Specify a new output because the default overwrites `results.json`. Compare hashes, model revisions, document order, numerical scores, gates and statistics; do not expect timestamps or timings to match. Record hardware or dependency changes rather than overwriting the old evidence.

## Expected observations

- Chunking: 32-character windows produce six chunks and fail all three model-plus-constraint checks. The three 64-character chunks retain voltage and capacity but not complete usage limits. The two 128-character chunks pass all three, as do the authored structured excerpts. This does not establish universal superiority of structured chunking.
- Q5: dense retrieval places the similar BR-6205 before BR-6204. Recall@3 is still 1, but RR is 0.5.
- Q1: RRF drops D1 from its top three; reranking brings it back. Fusion does not improve every question.
- Q2: nDCG@3 falls from 1 after fusion to approximately 0.920 after reranking.
- Q9: Chinese stress query. Lexical preprocessing retains only ASCII letters and numbers, and the models are English-focused; this is not a Chinese-search acceptance benchmark.
- Q10: warranty and price evidence is absent. Ranking metrics are null. Candidate retrieval is not answering or abstention evaluation.
- Wilson for 2/10 is approximately 5.7% to 51.0%. The synthetic paired change is about 33.3 percentage points; the six-question bootstrap interval is about 0 to 66.7 points, not proof of significance or causality.

Do not use synthetic product facts for procurement. The separately disclosed SuperPDR 17% and ELEREIN 15% historical case figures were not retested or recomputed here.

## Primary references

- [SQLite FTS5](https://www.sqlite.org/fts5.html)
- [Sentence Transformers](https://www.sbert.net/docs/quickstart.html)
- [Embedding model](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
- [Cross-encoder](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2)
- [JSON Schema](https://json-schema.org/learn/getting-started-step-by-step)

For learning and internal review. Production use needs domain data, independent annotation, access isolation, version governance and security review. No production service, automatic publication or performance guarantee is included.