AI项目在生产环境失败的真正原因：不是模型问题而是数据管道

Most of the AI content on this site is about models, prompts and frameworks. This one is about the layer underneath all of that, the part that decides whether the feature you shipped ever reaches a human.

我搭建并维护中小企业的自动化系统，而来自这些系统的 bug 报告几乎从不指向模型本身——它们指向的是那条每个人都以为早就工作正常的管道。

The Model Is Rarely The Bottleneck

When an AI feature disappoints in production, the instinct is to swap models. Sometimes that is the answer. Far more often the model did its job and something around it failed quietly.

当一个 AI 功能在生产环境中令人失望时，本能反应是换一个模型。有时候这确实是对的答案。但更多时候，模型已经完成了自己的工作，而它周围的东西却在悄然失效。

A support chatbot that answers confidently wrong is usually a retrieval problem, not a reasoning problem. It was handed the wrong three paragraphs, or the knowledge base has current pricing in one document and last year's pricing in another, and nothing in the system knows which one is authoritative.

一个自信地给出错误答案的客服聊天机器人，通常是检索问题，而非推理问题。它拿到了错误的三段内容，或者知识库里当前价格在一份文档里、去年的价格在另一份文档里，而系统里没有任何东西知道哪一份是权威的。

A prediction that misses is usually a collection problem. The training set has a field that got renamed six months ago and backfilled with nulls. No algorithm choice fixes that.

一个预测失误通常是大全问题。训练集里有一个字段在六个月前被重命名了，之后被回填为 null。算法选择再正确也解决不了这个问题。

The uncomfortable part is that all of these look like AI failures from the outside, which is why teams keep buying a better model instead of fixing the input.

令人不安的地方在于，所有这些从外部看起来都像是 AI 失败，这就是为什么团队一直在购买更好的模型而不是去修复输入。

Delivery Is An Infrastructure Problem

This is the one that catches people hardest, because it feels like it should be somebody else's job.

这是让人们栽跟头最深的一点，因为它让人觉得应该是别人的活儿。

If your feature sends email, the model's output is irrelevant until the message is accepted. That means SPF, DKIM and DMARC alignment on the sending domain, a warm-up ramp so a brand new domain is not suddenly pushing thousands of messages, volume shaping per receiving ISP, and bounce handling that actually suppresses addresses instead of retrying them straight into a blocklist. Get any of those wrong and beautifully written mail lands in a junk folder nobody opens.

如果你的功能要发邮件，那么在消息被接收之前，模型的输出毫无意义。这意味着发送域名要配置好 SPF、DKIM 和 DMARC 对齐，要有预热 ramping，让全新的域名不会突然推送成千上万条消息，要针对每个接收 ISP 做流量整形，以及要正确处理退信——真正地封禁地址而不是直接重试进 blocklist。上述任何一项没做对，精心撰写的邮件就会掉进没人打开的垃圾箱。

SMS is stricter. In the US you register the brand and the campaign under 10DLC before carriers will pass your traffic at real volume, and unregistered traffic gets filtered without a useful error coming back. None of that compliance work has anything to do with AI, and all of it decides whether the AI's output is ever read.

短信的限制更严。在美国，你需要在 10DLC 下注册品牌和营销活动，然后运营商才会以真实流量放行你的消息，未注册流量会被直接过滤，而且不会返回有用的错误信息。这些合规工作跟 AI 毫无关系，但它们决定了 AI 的输出是否会被阅读。

We keep our own notes on this in one place, twenty guides grouped by the job you are trying to do rather than by tool category, which is roughly the order these problems tend to show up in.

我们把关于这方面的笔记放在一处，二十篇指南按你想完成的任务分组而非按工具类别，大约是这些问题实际出现的顺序。

The pattern worth internalizing: every channel has a gatekeeper, and the gatekeeper does not care how good your content is.

值得内化到思维里的模式：每个渠道都有一个守门人，而守门人根本不在乎你的内容有多好。

Your Data Layer Sets The Ceiling On Answer Quality

Once delivery works, the next ceiling is what the model can actually see.

一旦投递工作了，下一个天花板就是模型实际上能看到什么。

Retrieval quality is mostly a chunking and freshness question. If documents get split mid-table or mid-procedure, the model receives fragments that read like contradictions. If nothing ever expires, stale answers outrank current ones forever.

检索质量主要是一个分块和新鲜度的问题。如果文档在表格中间或流程中间被切断，模型收到的片段读起来就像自相矛盾。如果没有任何东西会过期，过时的答案永远会排在最新答案前面。

The habit that pays for itself is logging the assembled context, not just the prompt template and the response. When an answer is wrong you want to know immediately whether the model reasoned badly or never saw the relevant document at all. Those are two different bugs with two different fixes, and without that log they are indistinguishable.

这个习惯回报丰厚：记录组装好的上下文，而不仅仅是 prompt 模板和响应。当答案错误时，你希望立刻知道是模型推理出了问题，还是压根没看到相关文档。这是两个需要不同修复的不同 bug，而没有那份日志，它们根本无法区分。

Same idea for tool calls. A 200 response is not a valid payload. Validate the shape before the next step plans on top of it, or you get an orchestrator confidently building on an empty array.

工具调用也是同样的道理。200 响应不等于有效载荷。在下一步基于它做规划之前，先验证一下数据形状，否则你就会得到一个自信地在空数组上构建的编排器。

Before you evaluate another model, check three things. Can the message actually be delivered. Does the system log what the model was given. Is there one authoritative source for the facts it answers from.

在你评估另一个模型之前，先检查三件事。消息是否真的能被投递。系统是否记录了给到模型的内容。模型回答问题所依据的事实，是否有一个权威来源。

If those three are solid, model upgrades compound. If they are not, a better model just produces better sounding wrong answers, faster.

如果这三项都稳固，模型升级会产生复利效应。如果不是，更好的模型只是更快地产生听起来更专业的错误答案。