Agent生产级部署的四大基础设施要件

Agents 在生产环境里缺的从来不是模型，而是四个基础设施原语：超时、重试与熔断、持久化状态、追踪。

---

Many teams moving agents from demo to production find that the first hurdle isn't model performance, but infrastructure. Running successfully once in a demo doesn't mean it can run day in and day out. We've helped many teams navigate this pitfall; what's usually missing are a few foundational setups.

许多团队把 agents 从 Demo 搬进生产环境时，发现第一道坎根本不是模型性能，而是基础设施。在 Demo 里跑通一次，不代表能日复一日地稳定运行。我们帮助过很多团队跨越这个陷阱——缺失的通常就是那么几项基础配置。

---

Timeouts, retries, state, and tracing as the production-ready toolkit

超时：别让一次慢 API 拖垮整条调用链

When an agent calls a tool and the other side doesn't respond, it can't wait indefinitely. Every call needs a timeout. When the time is up, it must either switch paths or throw an error. We once saw an agent where a half-dead third-party API stalled the main chain for 40 minutes. The user thought the system had crashed, but it was just stuck on a single call without a timeout.

当 agent 调用一个 tool 而对方没有响应时，它不能无限等下去。每次调用都需要设置超时。超时后必须切换路径或抛出错误。我们曾见过一个 agent，因为一个半死不活的第三方 API 把主流程卡了 40 分钟。用户以为系统崩溃了，实际上只是某次调用没有加超时，一直卡在那儿。

---

Retries and circuit breaking: back off on failure, trip the circuit on continuous failure

重试与熔断：失败后退避，持续失败则熔断

Occasional tool failures are normal, and a few automatic retries are fine. But if the same API fails continuously, you have to break the circuit. Don't stubbornly hammer a dead service. We've seen an agent exhaust its quota with dozens of retries just because a downstream service blipped for a second. The right approach is retrying with backoff, and tripping the circuit after a certain number of consecutive failures, then probing again later.

偶发的 tool 调用失败是正常的，自动重试几次也没问题。但如果同一个 API 持续失败，就必须熔断。别死命往一个已经挂掉的服务上撞。我们见过一个 agent，因为下游服务抖动了一秒，就疯狂重试了几十次，把配额耗光了。正确的做法是带退避的重试，在连续失败若干次后触发熔断，稍后再试探。

---

State: resuming from breakpoints after a restart

状态：机器重启后从断点恢复

If a machine restarts halfway through a long-running task, can it resume from the breakpoint instead of starting over? This requires the state of each step to be persisted, not just held in memory. If the state isn't persisted to disk, a restart means a complete rerun, and you simply cannot risk putting long-running tasks into production.

如果一台机器在执行一个长任务时重启了，能否从断点恢复而不是从头开始？这要求每一步的状态都要持久化，而不是只存在内存里。如果状态没有持久化到磁盘，重启就意味着全量重跑，这样的长任务根本不敢放生产。

---

Tracing: pulling up the entire chain when one step is slow

追踪：某一步慢了，把整条调用链拉出来看

When an agent executes over a dozen steps, you need to be able to pull up the entire execution trace to see exactly which step was slow or threw an error. Without tracing, debugging production issues is just guesswork. When onboarding customers to production environments, the very first thing we usually do is implement tracing. Once, a customer complained about a particularly slow response. We pulled up the trace and instantly saw that a retrieval tool was taking eight seconds. It wasn't a model issue, and troubleshooting took all of ten seconds. We are rolling out our free agent observability feature in waves now. TokenGo Agent Tracing.

当一个 agent 执行了十几步时，你需要能拉出完整的执行轨迹，精准看到是哪一步慢或抛了错。没有追踪，生产环境的问题排查就是瞎猜。帮客户接入生产环境时，我们通常第一件事就是上追踪。曾经有个客户抱怨响应特别慢，我们拉出 trace 一看，发现是一个 retrieval tool 花了 8 秒。根本不是模型的问题，排查全程不过 10 秒。我们现在正在分批推出免费的 agent 可观测性功能：TokenGo Agent Tracing。

---

Architecture: stateful orchestration layer + stateless tool layer

架构：有状态的编排层 + 无状态的 tool 层

None of these are difficult on their own. The hard part is that they must exist as a cohesive whole, ideally provided by the platform so that every business team doesn't have to reinvent the wheel. We favor a two-layer architecture: a stateful orchestration layer that remembers where the task is and can resume after restarts, and a stateless tool layer that can be spun up or swapped out at any time. If one dies, just swap it. The two layers communicate using the primitives mentioned above. If a tool goes down, the orchestration layer switches to another one and keeps running; the user on the other end might only wait an extra two seconds. We built a reconciliation agent for a financial client where the tool layer went down twice; both times, the orchestration layer switched to backups in seconds, and the business side was completely oblivious.

这些原语单独拎出来都不难。难的是它们必须作为一个整体协同运作，最好由平台统一提供，这样每个业务团队不用各自重复造轮子。我们倾向于两层架构：一层是有状态的编排层，负责记住任务做到哪儿了、重启后能续跑；另一层是无状态的 tool 层，随时可以拉起或替换。哪个挂了，换掉就是了。两层之间用上述原语通信。如果一个 tool 挂了，编排层切到另一个继续跑，用户那头可能只是多等了 2 秒。我们给一个金融客户做过一个对账 agent，tool 层挂了两次，两次编排层都在秒级切到了备份，业务侧完全无感知。

Stateful orchestration remembers the job. Stateless tools can be swapped.

有状态编排记住任务，无状态 tool 即换即用。

Taking agents to production is about whether it can pick itself back up after crashing. With a complete set of primitives, it can recover from crashes invisibly to the user.

把 agents 送上生产，关键在于它崩溃后能不能自己爬起来。有了完整的一套原语，就能在用户无感知的情况下从崩溃中恢复。

---

How this plays out in a real outage

真实故障中的表现

Let's look at a concrete example. In that reconciliation agent, the retrieval service in the tool layer crashed. The orchestration layer had a timeout set before the call; when it didn't get a response in eight seconds, it marked it as a failure and triggered the circuit breaker, stopping further requests to the dead service. The orchestration layer then swapped to a backup retrieval instance via an alternative path, recovering within ten seconds. The reconciliation task wasn't interrupted at all, and the business side noticed nothing. Looking at the trace afterward, the only anomaly in the entire chain was that the retrieval step jumped from its usual two seconds to an eight-second timeout while the rest proceeded normally. With the timeout + circuit breaker together, these two primitives turned an outage into a seamless switch.

来看一个具体例子。那个对账 agent 的 tool 层中，retrieval 服务挂了。编排层在调用前设了超时，8 秒没收到响应就标记为失败，触发熔断器，停止继续往那个已死服务发请求。然后编排层通过一条备用路径切换到了备份 retrieval 实例，10 秒内恢复。对账任务一点没被打断，业务侧毫无察觉。事后看 trace，整条调用链唯一的异常就是 retrieval 步骤从平时的 2 秒跳到了 8 秒超时，其余全部正常。超时 + 熔断器这两项原语联手，把一场故障变成了一次无感切换。

Timeout, circuit open, backup path: the task never stopped

超时触发、熔断开启、备用路径：任务从未中断

When we set up production environments for customers, the first thing we do is enable these four primitives by default. We don't leave it as an optional add-on. You don't think about these things during the demo phase, but if you wait until things break in production to patch them, the cost is immense.

给客户搭建生产环境时，我们第一件事就是默认把这四项原语全部启用。我们不把它当成可选附件。Demo 阶段你不会想到这些，但要是等到生产环境出事了才去补，代价是巨大的。

Internally, we call these four points the "Production-Ready Toolkit." When a new customer onboards, we first run a set of inspection scripts across their agent. Any missing primitive is flagged in red and prioritized for fixing, so they aren't left flying blind. Only when all four are green do we confidently say this agent is ready for production. The difference between running in a demo and running day in and day out is whether these four primitives are fully in place.

内部我们把这四点称为"Production-Ready Toolkit"（生产就绪工具包）。新客户接入时，我们先跑一套检查脚本，把缺失的原语标红并优先修复，不让他们蒙着眼飞。只有四项全部变绿，我们才敢自信地说这个 agent 可以上生产。Demo 能跑和日复一日稳定跑之间的差距，就看这四项原语有没有到位。

For another client, before going live, our scan revealed they were missing state persistence and tracing. After patching those, their average alert troubleshooting time dropped from two hours to ten minutes.

另一个客户在上线前我们的扫描发现他缺少状态持久化和追踪。补上之后，平均告警排查时间从两小时降到了十分钟。

Alert troubleshooting dropped from two hours to ten minutes

告警排查从两小时缩短至十分钟

We can perform production readiness checks for every business client we onboard. Whether they are missing timeouts, retries, state, or tracing, we fix it first. We also document the mechanics of these primitives for the customers to read. Here at TokenGo we believe trust comes from transparency.

我们可以为每一个接入的业务客户做生产就绪检查。不管缺的是超时、重试、状态还是追踪，都先补上。我们也会把这些原语的机制写成文档给客户阅读。在 TokenGo，我们相信信任来源于透明。