AI 写技术文档为何越写越错：模型在「幻觉文档」

An LLM asked to document a payments module — about 900 lines, four public functions, a retry wrapper — will typically produce something gorgeous. Structured headings, a parameters table, a "Common Pitfalls" section. It will also document a timeout_seconds parameter that does not exist. It will describe exponential backoff with jitter when the code does a flat time.sleep(2) in a for loop. It will include a usage example that imports a symbol from the wrong module.

Errors like these survive for weeks, because nobody reads documentation that nobody trusts. The first person to try the example files a bug against the library.

That's the core problem with pointing a model at a repo and asking for docs. The model isn't reading the code so much as pattern-matching it to the average of every similar library it has seen. A retry helper looks like tenacity, so it gets tenacity's parameters. A webhook handler looks like Stripe's, so it gets Stripe's idempotency semantics. The output is fluent, structurally correct, and wrong in exactly the places a reader can't check without reading the source — which is the whole reason they came to the docs.

Wrong docs are worse than no docs. No docs make you read the code. Wrong docs make you skip reading the code.

The reframe: transformation, not generation

Models are bad at inventing facts about a system and good at reformatting facts handed to them. So stop asking for documentation and start asking for transformations of artefacts that already encode the truth.

A codebase is full of these:

Type signatures. def charge(account_id: UUID, amount: Money, *, idempotency_key: str) -> ChargeResult already tells you every parameter, whether it's keyword-only, and what comes back. A model can't invent a fifth parameter if it's constrained to the signature.

Tests. A passing test is an example that ran. Turning test_refund_partial_amount into a documented example is a formatting job, not a reasoning job.

OpenAPI specs / JSON Schema. If the spec is generated from code (FastAPI, tsoa, utoipa), the spec is ground truth. Prose around it is the only thing to write.

Migrations. ALTER TABLE charges ADD COLUMN settled_at timestamptz NULL tells you the column exists, its type, and that it's nullable. That's three facts you don't have to trust anyone for.

Git history. git log --follow -p src/billing/retry.py explains why the sleep is flat. The commit message says "revert jitter, it broke replay determinism in tests." No model would guess that. Every model can summarise it once it's pasted in.

The prompt shifts from "document this module" to "here is the signature, here are the three tests that cover it, here is the commit that introduced it — write the docstring, and write UNKNOWN for anything not present in this input."

That last instruction does most of the work. Models will happily emit UNKNOWN when it's an allowed output. They will not volunteer uncertainty when the only available move is prose.

Work on diffs, not repos

A whole-repo docs pass is a one-time event that produces a large, unreviewable PR. Nobody reads a 4,000-line documentation diff carefully. It gets approved on vibes and rots from day one.

A diff-scoped pass is small enough to actually review, and it lands with the change that motivated it. A script along these lines does the job.

#!/usr/bin/env bash
# scripts/docstring-pass.sh — suggest docstrings for functions touched in this branch
set -euo pipefail

BASE="${1:-origin/main}"
OUT="${OUT:-.docsuggest}"
mkdir -p "$OUT"

# Changed source files, excluding tests and generated code
mapfile -t FILES < <(
  git diff --name-only "$BASE...HEAD" -- '*.py' \
    ':!tests/*' ':!*_pb2.py' ':!**/migrations/*'
)

[[ ${#FILES[@]} -eq 0 ]] && { echo "no source changes"; exit 0; }

for f in "${FILES[@]}"; do
  [[ -f "$f" ]] || continue

  # Tests that mention the module path — cheap but effective
  mod=$(basename "$f" .py)
  tests=$(grep -rl "$mod" tests/ 2>/dev/null | head -3 | xargs -r cat)

  {
    echo "## Current file: $f"
    echo '```

python'; cat "$f"; echo '

```'
    echo "## Diff in this branch"
    echo '```

diff'; git diff "$BASE...HEAD" -- "$f"; echo '

```'
    echo "## Relevant tests (these are the only examples known to run)"
    echo '```

python'; echo "$tests"; echo '

```'
    echo "## Recent history"
    git log --oneline -5 -- "$f"
  } | llm -m claude-sonnet-4-5 \
        --system "$(cat prompts/docstring.md)" \
        > "$OUT/$(echo "$f" | tr '/' '_').md"
done

echo "suggestions in $OUT/"

llm is Simon Willison's CLI (pipx install llm, then llm keys set anthropic). The system prompt is where the constraints live:

<!-- prompts/docstring.md -->
Write Google-style docstrings for functions ADDED or MODIFIED in the diff.
Do not touch functions that are unchanged.

Rules:
- Every parameter you document must appear in the function signature.
  Copy the name and type annotation exactly.
- Do not describe retry, caching, timeout, or concurrency behaviour
  unless it is visible in the code shown.
- Every code example must be derived from a test in the input.
  Cite the test name in a comment. If there is no test, write no example.
- If a parameter's meaning is not determinable from the code, tests,
  or commit messages, write: "UNKNOWN — needs author input".
- Output a unified diff against the current file. No commentary.

Output goes to a scratch directory, not straight into the file. The engineer applies what's right and deletes the rest. The UNKNOWN markers become a to-do list of things only a human knows.

The single highest-value check: every example in the docs must execute in CI. For Python, doctest gets you there almost free.

# .github/workflows/docs.yml
- name: Examples in docstrings must run
  run: python -m pytest --doctest-modules src/ -q

- name: Examples in markdown must run
  run: python -m pytest --codeblocks docs/   # pytest-codeblocks

- name: API docs must match the implementation
  run: |
    python -m app.export_openapi > /tmp/openapi.json
    git diff --exit-code --no-index docs/openapi.json /tmp/openapi.json

That last step is the one that catches drift. If someone adds a query parameter and doesn't regenerate the spec, CI fails with a diff showing exactly what changed. For contract-level checking, schemathesis run docs/openapi.json --url http://localhost:8000 will hammer a running service with requests derived from the spec and report where behaviour and documentation disagree.

Generated prose has no compiler

A docstring that describes behaviour removed six months ago will sit there forever. What's needed is a mechanical link between the doc and the thing it documents.

One approach is to stamp generated sections with a content hash of the source:

<!-- gen-from: src/billing/webhooks.py sha256:4f1a9c2e -->
### Webhook verification
...
<!-- /gen-from -->

# scripts/check_docstamps.py
import hashlib, pathlib, re, sys

PAT = re.compile(r"<!-- gen-from: (\S+) sha256:([0-9a-f]+) -->")
stale = []
for md in pathlib.Path("docs").rglob("*.md"):
    for src, want in PAT.findall(md.read_text()):
        got = hashlib.sha256(pathlib.Path(src).read_bytes()).hexdigest()[:8]
        if got != want:
            stale.append(f"{md}: {src} changed ({want} -> {got})")

if stale:
    print("Stale generated docs:\n  " + "\n  ".join(stale))
    sys.exit(1)

It's blunt — a whitespace change trips it — but a noisy check that forces a five-second re-read beats silent rot. Add python scripts/check_docstamps.py to CI and require it on protected branches.

Use the model as a reviewer instead

The highest-value prompt is rarely "write docs." It's this, run against a doc page plus the code it describes:

You are a competent engineer who has never seen this system. Read the documentation, then the code. List every point where the docs would leave you unable to complete the task, and every claim in the docs you cannot verify in the code. Do not rewrite anything.

Typical output from a run against a webhooks page:

"The page says signatures are verified but never says where the secret comes from — env var? config? I'd have to grep."

"The example shows a 200 response. What happens on a duplicate delivery? The code has an on_conflict_do_nothing that the docs don't mention."

"replay_window is documented in seconds. In the code it's compared against a timedelta built from minutes."

The third kind of finding is a real bug, not a documentation nit — surfaced without the model writing a word of documentation. Critique is a much easier task than generation, because the ground truth is in the context window instead of the weights.

Where this still doesn't work

Architecture docs. Anything requiring "why we chose this over that." Runbooks that depend on knowing which alert is a false alarm at 3am. Onboarding guides that need to know what a particular team finds confusing. The model has no access to the arguments in Slack, the outage that shaped the design, or the vendor limitation someone worked around. Generate these and you get a plausible-sounding history that never happened, which is a specific kind of poison for a new hire.

Also: models are bad at knowing what to leave out. Ask for a module overview and you'll get every private helper documented at equal weight to the one function anyone calls.

Whether anyone is actually reading

Producing docs is easy to measure and worthless. These signals are better.

Search queries with zero results. If the docs site has search (Algolia DocSearch, or MkDocs Material's built-in with the plugin's log), the null-result queries are a direct list of pages that should be written. If there are no null-result queries, nobody is searching.

Support questions that have an answer in the docs. Tag them. If a question is answered on a page and someone asked anyway, the page exists but doesn't surface, doesn't rank, or isn't believable. That's three different fixes.

Links in code review. Grep PR comments for the docs domain. Engineers linking each other to a page is the strongest signal that the page is load-bearing.

A canary. Bury one specific, checkable detail in a page — a named constant, an unusual flag. When someone uses it correctly without asking, you know they read it.

Time-to-first-merged-PR for new hires. Slow to move, but it's the number the docs exist to change.

Teams that look honestly at page views over a quarter routinely find a third of their documentation can be deleted without anyone noticing. What deserves to survive is the set of pages updated in the same PR as the code, with examples that run in CI, and a model doing the first pass on the docstrings and a second pass as the confused reader. That combination is worth real engineering time. Pointing the model at the repo and pressing go is not.