LLM调用成本无法提前精确预估：实测揭示估算偏差高达61倍

我花了一周时间为 AI agent 构建支出上限功能，基于一个后来证明是错误的假设，而这个假设错误的方式比这个功能本身更有意思。

The assumption is the obvious one. Before making a model call, estimate what it will cost. If that estimate breaks the budget, refuse the call. Every budget system works this way: check, then spend.

这个假设显而易见：在调用模型之前，先估算成本。如果估算结果超出预算，就拒绝调用。每个预算系统都是这么工作的：先检查，再支出。

Then I measured the estimate against real calls, and it does not work. Not "needs tuning". Does not work, and cannot be made to.

然后我把估算结果和真实调用进行了对比，发现它根本不起作用。不是"需要调优"的那种不行——是完全不起作用，而且无法通过调优来解决。

12 calls recorded through OpenRouter against openai/gpt-oss-20b:free, 9 of which had a comparable pre-call estimate. For each, I compared what the estimator predicted against what the vendor's own usage block reported afterwards.

我在 OpenRouter 上记录了 12 次针对 openai/gpt-oss-20b:free 的调用，其中 9 次有可对比的预调用估算。我将估算器的预测结果与供应商后续报告的使用量数据进行了逐项对比。

Then I replayed those 9 calls against a $0.005 per-call cap and asked how many the estimator would have refused, where the real cost turned out to be under the cap:

然后我用 $0.005/次 的上限重新回放这 9 次调用，问自己估算器会拒绝多少次——在这些被拒绝的调用中，实际成本其实都在上限之内：

4 of 9 legitimate calls wrongly refused.

9 次合法调用中有 4 次被错误地拒绝了。

At a $0.05 cap, nothing was refused at all. Which sounds like the fix, until you look at which calls were underestimated.

当上限设为 $0.05 时，没有任何调用被拒绝。这听起来像是解决方案，直到你看看哪些调用被低估了。

Both directions is what kills it

双向偏差才是致命问题

If the estimate were consistently high, you would divide by a constant and move on. If it were consistently low, you would multiply.

如果估算结果一直偏高，你可以除以一个常数来调整，然后继续。如果一直偏低，你可以乘以一个系数。

It is neither. The two calls in my sample that did not declare max_tokens came in 18x and 22x too low.

但实际情况两者都不是。我样本中有两次调用没有声明 max_tokens，低估了 18 倍和 22 倍。

That is the whole problem in one sentence. Tighten the cap and it refuses ordinary work. Loosen it until it stops doing that, and it waves through precisely the unbounded calls a cap exists to stop, because those are the ones it underestimates.

这就是问题的全部。收紧上限就会拒绝普通工作；放宽到不再拒绝普通工作时，它又会放行那些上限本应拦截的无界调用——因为这些正是被低估的调用。

The estimator is most wrong exactly where being wrong is most expensive.

估算器犯错的代价最高的时刻，恰好是它犯错最严重的时候。

Why it is unfixable, not undertuned

为什么这是无解的，而不是没调好

Input tokens are knowable. You have the prompt, you can count it.

输入 token 是可知的。你有提示词，就能数出来。

Output tokens are not knowable. That is not a gap in the estimator, it is a property of generation: the model decides how much to write while it is writing. "Summarise this" can return one line or four paragraphs, and the difference is 40x in cost.

输出 token 是不可知的。这不是估算器的缺陷，而是生成的本质属性：模型在生成过程中决定写多少。"总结这段文字"可能返回一行，也可能返回四段，成本相差 40 倍。

No overhead constant fixes an unknown that varies 60x in one direction and 22x in the other. No rolling percentile fixes it either, because the p95 of a distribution with that spread is either useless as a limit or useless as a prediction.

一个固定的开销常数无法修正这个在一个方向上变化 60 倍、在另一个方向上变化 22 倍的不确定值。滚动百分位数也无法解决，因为这种分布的 p95 要么作为限制毫无意义，要么作为预测毫无意义。

People do reach for max_tokens here, and it does bound the worst case. But it bounds it at a ceiling nobody sets tightly, because setting it tightly truncates real answers. A ceiling of 4096 on a call that returns 200 tokens prices at 20x reality, which is the 61x row above.

人们确实会在这里求助 max_tokens，它确实能限制最坏情况。但它限制的是一个没人会紧密设置的天花板，因为紧密设置会截断真实答案。在返回 200 个 token 的调用上设置 4096 的上限会将实际成本高估 20 倍。

Enforce against spend that has already been reconciled.

根据已对账的支出来执行限制。

After each call, the vendor tells you exactly how many tokens it used. Price that against a rate table and add it to a running total. When the total crosses the cap, stop.

每次调用后，供应商会精确告知使用了多少 token。将这个数字对应费率表计算后加入累计总额。当总额超过上限时，停止。

Reconciliation error in the same measurement: 0. Not 0.1%. The vendor tells you the number, so there is nothing to estimate.

对账误差：0。不是 0.1%。供应商直接告诉你数字，所以根本不需要估算。

The tradeoff is real and worth stating plainly: you can overshoot by one call. The cap is checked before a call using spend up to that point, so the call that crosses the line still completes. You are trading exactness for the ability to work at all.

这个权衡是真实存在的，值得直说：你可以超支一次调用。在调用前检查的是到那时为止的支出，所以跨越红线的调用仍会完成。你是用精确性换取持续工作的能力。

That is a good trade, because "one call of overshoot" is bounded and knowable, and "refuses 44% of legitimate work while missing the runaways" is neither.

这是个划算的权衡，因为"超支一次调用"是有界限的、可知的，而"拒绝 44% 的合法工作同时漏掉失控调用"则两者都不是。

The one place the estimate still earns its keep

估算仍有用的一个场景

If a single call could cost more than some absolute number, refuse it before making it. Here the estimate is fine, because:

如果单次调用可能超过某个绝对数值，就在调用前拒绝它。这里估算没问题，因为：

max_tokens is declared in this case, so the ceiling is real rather than guessed

这种情况下 max_tokens 是声明的，所以上限是真实的而非猜测的

the threshold sits orders of magnitude away from typical, so a 3.87x error does not change the answer

阈值与典型成本相差数个数量级，所以 3.87 倍的误差不会改变判断

A $50 per-call ceiling is not troubled by an estimator that is 4x off on a call that costs $0.02. The failure mode above only appears when the threshold is close enough to typical spend that a 4x error crosses it.

$50 的单次调用上限不会被 4 倍误差的估算器影响——即便某次调用实际成本只有 $0.02。上述失败模式只会在阈值接近典型支出、4 倍误差就能跨越阈值时才会出现。

Caveats, before someone else supplies them

在别人提出之前，先说明一下局限性

Nine comparable calls is a small sample. Two unbounded calls is a tiny one. It was all one model on one provider.

9 次可对比的调用是小样本，2 次无界调用更是微乎其微。这全部来自一个模型和一个供应商。

I would take this as directional rather than conclusive, and I would be genuinely interested if anyone has measured it at larger scale — I could not find anyone who had, which is partly why I am writing it down. The failure was consistent and bidirectional across every call I looked at, and the mechanism (output length being unknowable in advance) is not specific to a model or a provider.

我会把这个视为方向性结论而非最终定论，如果有人在大规模上做过测量，我会很感兴趣——我找不到任何这样的测量，这是我写这篇文章的部分原因。失败在我查看的每次调用中都一致地、bidirectional 地出现，而失败的原因（输出长度无法提前知晓）并不是特定模型或供应商的问题。

focxle is what came out of it: a Python package that tells you what each of your AI agents cost across every vendor it touched, model calls and HTTP APIs together.

focxle 就是从这个项目中产生的：一个 Python 包，可以告诉你每个 AI agent 在每个供应商处的成本，包括模型调用和 HTTP API。

pip install focxle

import focxle
focxle.init()

It prints per agent, per vendor, when the process exits. It also shows what a cap would have stopped, using the reconciled approach above, so you can see whether you want one before turning one on.

它在进程退出时打印每个 agent、每个供应商的成本。它还展示了一个上限会拦截什么，使用上面那种对账方法，这样你可以在开启之前看到是否需要它。

Zero dependencies, no account, no proxy in front of your traffic, and no network calls at all on the free path. MIT, source at github.com/focxle/focxle-python.

零依赖，无需账号，不在流量前加代理，免费路径上完全不产生网络调用。MIT 许可证，源代码在 github.com/focxle/focxle-python。

Mostly though I wanted the negative result written down somewhere. The estimate-then-refuse design is the obvious one, it is what I would have built again, and it fails in a way you only see if you go and measure it.

不过我主要是想让这个负面结果被记录下来。先估算后拒绝的设计是显而易见的，我下次还会这么构建，但它以一种只有在实际测量后才能看到的方式失败了。