同一发票集在 OpenRouter、直连 OpenAI 和 Claude API 三条路线上跑,评估每条路线的 accepted invoice 数量与 token 成本;强调用实际生产级样本(含多页发票、旋转扫描、缺订单号等)建立冷冻评估语料库。
Short answer: no public price table can establish whether OpenRouter, direct OpenAI, or Claude API access is cheapest for a SaaS app; compare each route by cost per accepted supplier invoice within the same latency budget. Keep one tested fallback behind your own TypeScript interface; a cheap token is irrelevant when a malformed total, a missed currency, or a slow retry forces another call.
简短回答:没有任何公开价目表能够断定 OpenRouter、直连 OpenAI 还是 Claude API 对 SaaS 应用最划算;应在相同的延迟预算下,比较每条路径接受一张供应商发票的成本。在你自己的 TypeScript 接口背后保留一条经过测试的备用路径;当一张格式错误的合计金额、一种遗漏的货币、或一次缓慢的重试导致再次调用时,低价 token 毫无意义。
For invoice extraction, I wouldn't select between OpenRouter, direct OpenAI, and the Claude API from a public price table alone. Run the same redacted invoice set through each route, validate the same fields, and count every attempt. A unified key can reduce integration work; direct accounts can keep provider-specific controls visible. Neither property settles the quality-versus-latency decision.
对于发票提取,我不会仅凭公开价目表就在 OpenRouter、直连 OpenAI 和 Claude API 之间做选择。用同一份脱敏发票集走每条路径,验证相同的字段,并统计每一次尝试。统一密钥可以减少集成工作量;直连账户可以保持提供商特定控制的可视性。两种特性都不能解决质量与延迟之间的权衡问题。
The practical unit is an accepted invoice.
实际单位是一张被接受的发票。
Should a Node.js SaaS app compare direct API routes with a unified key?
Node.js SaaS 应用应该比较直连 API 路径与统一密钥吗?
Start with a frozen evaluation corpus that resembles production, including the documents that make extraction awkward: multi-page invoices, credit notes, repeated tax rows, comma decimal separators, missing purchase-order numbers, and scans with rotated pages. Keep tenant data out of a shared benchmark unless its handling and retention are explicitly approved. The point is to compare routes against identical inputs, not to reward whichever route received the easy documents that week.
从一个贴近生产环境的固定评估语料库开始,包含那些让提取变得棘手的文档:多页发票、贷项通知单、重复的税行、逗号小数分隔符、缺失的采购单号、以及页面有旋转的扫描件。除非数据处理和保留已获得明确批准,否则不要将租户数据放入共享基准测试。目的是在相同输入下比较各条路径,而不是让某条路径因为当周收到了简单文档就获得奖励。
Define acceptance before sending a request. For a supplier invoice, a useful contract might require supplierName, invoiceNumber, invoiceDate, currency, subtotal, tax, and total, plus line-level evidence for fields that drive payment. Syntax is the shallow gate. Semantic checks should reject impossible dates, currencies outside the tenant's policy, duplicate invoice numbers, and totals that do not reconcile within a declared rounding tolerance. Human review is still appropriate for ambiguous documents; the evaluator should record that outcome rather than quietly treating it as model failure.
在发送请求之前先定义接受标准。对于供应商发票,有用的契约可能要求 supplierName、invoiceNumber、invoiceDate、currency、subtotal、tax 和 total,加上驱动付款的字段的逐行证据。语法是浅层门槛。语义检查应拒绝不可能的日期、超出租户策略的货币、重复的发票号、以及在声明的四舍五入容差内无法核对的合计金额。对于模糊的文档,人工审核仍然是适当的;评估者应记录该结果,而不是悄悄将其视为模型失败。
Then measure three clocks separately: time to first usable response, time through validation, and time to final acceptance after any fallback. A route can look quick at the HTTP boundary yet lose once invalid output triggers a second attempt. Conversely, a slower first pass can win for an interactive workflow if it avoids review often enough. Your mileage may vary because invoice mix, prompt shape, output schema, and model revisions all move the result.
然后分别测量三个时钟:首次可用响应时间、通过验证的时间、以及任何备用方案后的最终接受时间。一条路径在 HTTP 边界可能看起来很快,但一旦无效输出触发第二次尝试就会失利。相反,对于交互式工作流,较慢的首轮通过可能因足够频繁地避免审核而胜出。你的结果可能不同,因为发票组合、提示词形状、输出 schema 和模型版本更新都会改变结果。
OpenRouter, direct OpenAI, and the Claude API belong in this experiment as routes, not as conclusions. Capture a dated model identifier and a dated pricing snapshot for each candidate. Don't assume models with similar labels have interchangeable behavior, and don't combine results from different prompts into one league table. The cheapest route is the one with the lowest observed accepted-result cost under the same contract; without that workload evidence, I'm not sure a universal cheapest claim means anything.
OpenRouter、直连 OpenAI 和 Claude API 在这个实验中应作为路径而非结论来对待。为每个候选方案记录一个带日期的模型标识符和带日期的价格快照。不要假设标签相似的模型具有可互换的行为,也不要将不同提示词的结果合并到同一张排行榜中。最便宜的路径是在相同契约下观察到的被接受结果成本最低的那条;没有这份工作负载证据,我不确定"全局最便宜"这个说法有什么意义。
Use a compact ledger for each attempt:
为每次尝试使用一份紧凑的账本:
Don't average away the tail. Report median and high-percentile acceptance latency, accepted-result rate, review rate, and cost per accepted invoice. Slice those numbers by document class. Ten clean digital PDFs can hide one dense scanned credit note, yet that one note may be exactly where an automatic payment system needs the stricter gate.
不要用平均数抹掉尾部。报告中位数和高百分位接受延迟、被接受结果率、审核率以及每张被接受发票的成本。按文档类别细分这些数字。十份干净的数字化 PDF 可能掩盖一张密集扫描的贷项通知单,而那张通知单可能恰恰是自动支付系统需要更严格门槛的地方。
Invoice quality is the first gate
发票质量是第一道门槛
The simple approach is to parse JSON, check that keys exist, and route failures to the next model. It fails because plausible strings are not necessarily correct accounting data. "total": "1,234.50" might be valid for one locale, while "1.234,50" needs another parser; accepting both as arbitrary strings only postpones the error. The chosen approach should normalize deterministically, validate arithmetic, and return a small set of outcomes that the router understands.
简单的方法是解析 JSON、检查键是否存在,然后将失败路由到下一个模型。它会失败,因为合理的字符串不一定是正确的会计数据。"total": "1,234.50" 可能对一种语言区域有效,而 "1.234,50" 需要另一种解析器;将两者都作为任意字符串接受只是推迟了错误。所选方法应该确定性归一化、验证算术,并返回路由器能理解的一小部分结果。
This focused TypeScript example scores already-normalized model output. It makes no network request, so the same gate can sit behind a direct integration or a unified-key integration.
这个聚焦的 TypeScript 示例对已经归一化的模型输出进行评分。它不发出网络请求,因此相同的验证门可以位于直连集成或统一密钥集成之后。
type InvoiceCandidate = {
invoiceNumber?: string;
currency?: string;
subtotalMinor?: number;
taxMinor?: number;
totalMinor?: number;
};
type Verdict =
| { accepted: true }
| { accepted: false; reason: "missing_field" | "unsupported_currency" | "total_mismatch" };
const allowedCurrencies = new Set(["USD", "EUR", "GBP"]);
function validateInvoice(candidate: InvoiceCandidate): Verdict {
const { invoiceNumber, currency, subtotalMinor, taxMinor, totalMinor } = candidate;
if (
!invoiceNumber ||
!currency ||
subtotalMinor === undefined ||
taxMinor === undefined ||
totalMinor === undefined
) {
return { accepted: false, reason: "missing_field" };
}
if (!allowedCurrencies.has(currency)) {
return { accepted: false, reason: "unsupported_currency" };
}
if (Math.abs(subtotalMinor + taxMinor - totalMinor) > 1) {
return { accepted: false, reason: "total_mismatch" };
}
return { accepted: true };
}
Minor currency units avoid a floating-point comparison in the gate, while the one-unit tolerance makes the rounding policy explicit. Real invoices need more rules: negative totals for credit notes, jurisdiction-specific tax treatment, and line-item reconciliation. Those rules belong in versioned domain code. They shouldn't be buried in a prompt where a copy edit can change payment behavior.
使用货币最小单位可以避免在验证门中进行浮点数比较,而一单位的容差使舍入策略明确。真实发票需要更多规则:贷项通知单的负合计、特定司法管辖区的税务处理以及逐行核对。这些规则应属于版本化的领域代码。它们不应被埋藏在提示词中,因为一次文案编辑就可能改变支付行为。
A failed validation is not automatically permission to call another model. Classify it. A truncated response may justify a controlled retry; an unsupported currency is a policy decision; unreadable source pixels may require document review. Blind fallback can pay twice for an input that no model should approve. Keep the original candidate, validation reason, and subsequent decision linked by one correlation ID, but do not put raw supplier details into routine logs.
验证失败并不自动获得调用另一个模型的许可。要对其进行分类。截断的响应可能有理由进行受控重试;不支持的货币是策略决策;无法读取的源图像像素可能需要文档审核。盲目备用可能为一个任何模型都不应批准输入付出双倍代价。用一个关联 ID 将原始候选对象、验证原因和后续决策链接在一起,但不要将原始供应商详情放入常规日志。
This is also where the quality-versus-latency policy becomes concrete. An upload workflow may return processing and finish asynchronously, while an operator waiting to approve a bill may need a hard deadline. Set those product budgets first. Then let the router enforce them.
这也是质量与延迟策略变得具体的地方。上传工作流可能返回"处理中"并异步完成,而等待批准账单的运维人员可能需要一个硬截止日期。先设定这些产品预算。然后让路由器强制执行它们。
Retry semantics define the latency ceiling
重试语义定义延迟上限
Retries need HTTP semantics, not optimism. RFC 9110 distinguishes idempotent methods because an automatic retry can repeat an intended effect when the client does not know whether the first request succeeded. Model inference is often invoked with POST; even if generating text has no physical side effect, your surrounding code may reserve quota, append an attempt, or enqueue review. Give each logical extraction an idempotency key inside your application and make ledger writes conditional on that key.
重试需要 HTTP 语义,而非乐观估计。RFC 9110 区分了幂等方法,因为当客户端不知道第一个请求是否成功时,自动重试可能重复预期效果。模型推理通常用 POST 调用;即使生成文本没有物理副作用,周围代码可能预留配额、追加尝试、或将审核加入队列。在应用程序内部为每个逻辑提取赋予一个幂等键,并使账本写入以该键为条件。
A 429 should not become an instant spray across every configured route. Respect an applicable retry delay, add bounded jitter, and stop when the remaining latency budget cannot fit another attempt. A connection timeout is ambiguous — the provider may have processed the request even though the client did not receive the response — so deduplicate your own accounting and downstream actions. Never let a late primary response and a fallback response create two payable invoice records.
429 不应变成对每条配置路径的即时喷洒。遵守适用的重试延迟,添加有界的抖动,并且当剩余延迟预算无法容纳另一次尝试时停止。连接超时是模糊的——提供商可能已处理了请求,即使客户端未收到响应——因此要对自身的账务和下游操作进行去重。永远不要让延迟的主响应和备用响应产生两张可付款的发票记录。
The state machine can stay small: pending, attempting, validating, accepted, review, and failed. Store route policy separately from extraction logic. The policy can say that a high-priority tenant gets one alternate route after a retryable transport outcome, while a batch import goes to review after a semantic mismatch. This keeps provider churn out of the invoice domain and makes a unified key an implementation option rather than an architectural dependency.
状态机可以保持精简:pending、attempting、validating、accepted、review 和 failed。将路由策略与提取逻辑分开存储。策略可以规定:高优先级租户在遇到可重试的传输结果后获得一条备用路由,而批量导入在遇到语义不匹配后进入审核。这将提供商更替排除在发票领域之外,并使统一密钥成为实现选项而非架构依赖。
There is a catch. Hedged requests can reduce tail latency by starting a second route before the first finishes, but they increase token use and complicate cancellation and accounting. They are not suitable when the job is asynchronous or the value of a few saved seconds is lower than a second inference attempt. Stick with one bounded attempt plus queued fallback for bulk imports. For a genuinely interactive approval screen, test hedging against a strict delay and count both calls, including responses you discard.
但有一个问题。对冲请求可以通过在第一个完成之前启动第二条路由来减少尾部延迟,但它们会增加 token 使用量,并使取消和账务复杂化。当作业是异步的,或者节省几秒钟的价值低于第二次推理尝试时,它们就不适用了。对于批量导入,坚持使用一次有界的尝试加排队备用。对于真正的交互式审批屏幕,用严格延迟测试对冲,并统计两次调用,包括你丢弃的响应。
Batch processing is another distinct lane. The OpenAI Batch API guide describes asynchronous processing with a completion window, so it can fit nightly backfills or non-interactive imports rather than a user waiting on an approval screen. Treat that as a workload split, not a blanket recommendation: the latency contract changes, and the benchmark must label batch results separately from synchronous ones.
批处理是另一条独立的通道。OpenAI Batch API 指南描述了带完成窗口的异步处理,因此它适合夜间数据填充或非交互式导入,而不是用户等待审批屏幕。将这视为工作负载拆分,而非一刀切的建议:延迟契约会发生变化,基准测试必须将批处理结果与同步结果分开标记。
Operationally, alert on changes in accepted-result rate, review rate, fallback frequency, and latency by corpus class. Token totals alone won't tell you that a prompt revision started dropping tax identifiers. Run a small frozen canary set before changing a model snapshot or validation prompt, and preserve the old policy long enough to compare. The route with the attractive first-call result may stop looking attractive once its review queue is visible.
在运维层面,按语料类别对被接受结果率、审核率、备用频率和延迟的变化发出警报。仅看 token 总数无法告诉你提示词修订开始丢失税标识符。在更改模型快照或验证提示词之前,运行一小份固定的金丝雀集,并保留旧策略足够长时间以进行比较。具有吸引人的首轮调用结果的路径一旦其审核队列可见,可能就不再具有吸引力了。
Read the evaluation as a deployment experiment
将评估作为部署实验来解读
Measure at least one full billing cycle if invoice seasonality matters, but don't wait for a perfect study before shipping a guarded version. Begin with a redacted stratified sample, a schema version, an explicit review path, and a hard cap on attempts. Record provider-reported token counts when available and keep the raw rate card version used for cost calculation. Recompute historical ledger rows when you want to test a new price assumption; don't rewrite the original usage facts.
如果发票季节性重要,至少测量一个完整的计费周期,但不要在有充分研究之前就发布一个有防护的版本。从一份脱敏的分层样本、一个 schema 版本、一条明确的审核路径和一次尝试的硬上限开始。在可用时记录提供商报告的 token 计数,并保留用于成本计算的原始价目表版本。当你想测试新的价格假设时,重新计算历史账本行;不要重写原始使用事实。
The decision record should state the latency budget, minimum accepted-result rate, permitted review rate, maximum attempts, data-handling constraints, and exit condition. It should also identify the switching cost. A provider-neutral TypeScript interface reduces code changes, but prompts, model behavior, safety controls, regional availability, and contract terms can still create lock-in. A unified key reduces credential handling inside the app, yet it adds another policy and trust boundary. Direct integrations expose separate credentials and operational surfaces. Those are engineering costs even when they don't appear in a token column.
决策记录应说明延迟预算、最低接受结果率、允许的审核率、最大尝试次数、数据处理约束和退出条件。它还应识别切换成本。提供商中立的 TypeScript 接口减少了代码更改,但提示词、模型行为、安全控制、区域可用性和合同条款仍可能造成锁定。统一密钥减少了应用内的凭证处理,但增加了一个额外的策略和信任边界。直连集成暴露了独立的凭证和运维层面。这些都是工程成本,即使它们没有出现在 token 列中。
For a solo team, the sensible first release is narrow: one primary route, one fallback condition, one deterministic validator, and one dashboard based on accepted invoices. Expand only when the ledger shows a real failure mode. More routes create more combinations to test, more model changes to watch, and more ways to spend money twice.
对于独立团队,合理的首发版本应该是狭窄的:一条主路由、一个备用条件、一个确定性验证器、以及一个基于被接受发票的仪表板。只有当账本显示出真正的失败模式时才扩展。更多路由会产生更多要测试的组合、更多要关注的模型更改、以及更多双倍花钱的方式。
The final choice should be reversible. Keep invoice validation and the attempt ledger under your control, pin candidate identities, and rerun the frozen corpus when quality, latency, or terms change. Use the measured frontier rather than a universal ranking: retain any route only while it meets the product's acceptance threshold and latency limit at an acceptable complete cost, including retries and human review.
最终选择应该是可逆的。将发票验证和尝试账本置于你的控制之下,固定候选身份,并在质量、延迟或条款发生变化时重新运行固定语料库。使用经过测量的前沿而非通用排名:仅在路径满足产品的接受阈值和延迟限制且包括重试和人工审核在内的完整成本可接受时保留它。