MCP服务器设计陷阱：描述文本与模型理解的Gap

Every MCP server I have written passed its tests before it was any good.

That is not a joke about test quality. The tests were fine. list_tools returned the right shape, every handler round-tripped its arguments, the JSON-RPC framing was correct, errors serialised. Green across the board. And then the model on the other end would call search(query="...", max_results=5) when the parameter was named limit, or call the tool three times in a row with identical arguments because the first response didn't obviously say it had succeeded, or never call it at all.

None of that is a bug in the server. All of it is a defect in the product. This is the thing I keep having to re-learn: an MCP server's real interface is not the wire protocol. It is the description text, and nothing in your test suite touches it.

Look at what a model actually receives from your server. Not the code — the payload:

• the one-line description
• the JSON Schema, including each property's description
• whatever your last response said

That's it. There is no README in the context window. There is no example usage unless you put it in the description. The model is integrating against your API having read exactly the fields above, once, in a list alongside forty other tools.

Now look at what your tests assert. Almost certainly: given arguments X, the handler returns Y. Which is the one part of the system that was never in doubt.

The gap between those two lists is where all my time has gone.

Three defects that pass every test

The name that lies slightly. I had a tool called get_status. It returned the status of a job, not of the server. Perfectly documented in the schema description. The model called it whenever it wanted to know if anything was healthy, because the name is what gets pattern-matched under load and the description is what gets skimmed. Renaming it to get_job_status fixed a class of misuse that no amount of description prose had fixed. Names are load-bearing. Treat renaming as a real fix, not cosmetics.

The optional parameter that isn't. A schema said path was optional, defaulting to the workspace root. Sensible. In practice the model omitted it constantly and got back a 400-item listing that ate the context window, and then apologised and tried again. The schema was honest and the default was wrong. "Optional" in JSON Schema means the call is valid without it. It does not mean the call is useful without it, and the model has no way to tell those apart from the schema alone. Either make it required or make the default genuinely cheap.

The success that reads like a failure. A write tool returned {"ok": true}. The model would frequently re-issue the write. Why wouldn't it? The response contained no evidence that anything had happened — no path, no byte count, nothing to quote back to the user. Returning Wrote 1,204 bytes to /home/x/notes.md stopped the retries immediately. Responses aren't just control flow; they're what the model reasons over on the next turn. A response that can't be reasoned over gets retried.

Notice that all three are content defects living in string fields. Type-checked, schema-valid, and wrong.

What I do instead now

I have stopped thinking of this as testing and started thinking of it as evaluation, which is an uncomfortable admission because evaluation is slower, fuzzier and harder to put in CI. But the alternative is shipping a server that is provably correct and practically unusable.

Read the tool list as the model sees it. Dump exactly what goes over the wire in tools/list and read it cold, as a flat list, with no knowledge of your codebase. Every time I do this I find two tools whose descriptions only make sense if you already know which one you want — which is the one thing the reader doesn't know.

Keep a small set of intent transcripts. Not "call tool X with args Y" but "given this user request, did the model reach the right tool on the first call?" Ten of these, run by hand before a release, catch more than a hundred handler assertions. They are noisy and non-deterministic. They are also the only tests that exercise the actual interface.

Assert on descriptions in CI. This part is mechanisable, and it's the highest value-per-line test I write: every tool has a non-empty description; every property has a non-empty description; no description is under N characters; names match a convention. It catches nothing subtle and it catches the boring regression where someone adds a parameter and doesn't document it, which is the most common way a good server decays.

Treat repeated calls as a bug report. When I see a model call the same tool twice with the same arguments, I no longer read it as the model being careless. It is nearly always my response failing to convey that the work was done. The model is the smoke detector; the fire is in my output formatting.

The uncomfortable part

There's an instinct — I have it strongly — to treat the model's misuse of a correct API as the model's problem. It's a defensible position and it is also a losing one, because you can't file a bug against the caller. The caller is a probabilistic system that read your schema once. If it consistently misreads you, the schema is ambiguous, whatever the type checker thinks.

The mental shift that helped: your MCP server is not a library. It's closer to a CLI designed for someone who will never read --help twice and cannot ask you a question. Everything that person needs must be legible at the moment of the call. That's a documentation problem wearing a protocol's clothes, and it will not show up green or red in your test runner.

I write about agent tooling and the boring engineering underneath it. Some of this material is worked through in more depth in my book, Building Production MCP Servers.