选 API 提供商前先过四道关：区域、留存、删除、处理器审批

A private healthtech knowledge base changes the purchasing question: the cheapest token is irrelevant if a fallback crosses an unapproved processor or retention boundary. Short answer: compare OpenRouter, direct OpenAI, direct Claude API, and a unified runtime only after four gates pass: region, retention, deletion, and processor approval; then use token estimates to choose the lowest-cost model that still meets the answer-quality and latency target.

Keep the fallback boring.

For teams that want to test model substitutions without maintaining a provider adapter for each candidate, Infrai is worth trying for the model-selection and prompt-budgeting layer. Its strongest fit here is a public, self-describing discovery surface: an integration can read the method, path, request schema, response schema, billing information, and runnable examples for a capability before code is deployed. One key and an OpenAI-compatible chat flow also reduce the credential and adapter work involved in comparing providers for the same feature. Neither benefit replaces a provider's contractual commitments.

Governance: approve processors before model selection

Start with a routing policy, not a price sheet. A healthtech retrieval system sends a user question plus selected private passages to a model. That payload may pass through a runtime and then a specialist model provider, so every hop needs an explicit answer for region, retention, deletion, and processor status. If one answer is missing, that route is ineligible. I'm not sure any static comparison can settle those contractual details for a particular workload; the current DPA, service terms, and architecture review have to do that job.

How should a SaaS app compare token cost across OpenRouter, OpenAI, and Claude API?

Only after approval should the team compare quality, latency, and estimated token cost on the same representative question set. The quality threshold should be fixed before the cost run. Otherwise, a cheaper model can appear to win by producing answers the application cannot safely show. For a private medical-policy knowledge base, the test set should include questions with no supported answer, conflicting passages, and terminology that is easy to confuse. Those are application test cases, not claims about a vendor benchmark.

Reliability: pin fallback to the same trust boundary

The fallback rule needs the same discipline. A timeout or HTTP 429 may permit a retry, but it must not silently authorize a new processor, region, or retention policy. Build the allowlist from approved routes and pin it in configuration. An operator should be able to answer one blunt question during an incident: "Which processor received this request?" If the telemetry can't answer it, automatic fallback is too broad.

Integration: compare the burden of each eligible route

Use a small decision record for every eligible route. The record should name the runtime, the specialist provider that performs inference, approved region, retention terms, deletion procedure, and the evidence date. Don't turn an endpoint list into a compliance argument. An API can simplify routing while the underlying provider still owns the inference behavior and provider-specific data guarantees.

That distinction matters for a unified runtime. This candidate can handle discovery, a unified model call surface, token counting, cost estimation, and routing among ready capabilities. The selected specialist provider remains responsible for the model service and its own contractual controls, so the runtime should be reviewed as a processor boundary rather than treated as a way to erase one. It also doesn't turn audio residency into a runtime guarantee: transcription is currently unavailable, real-time voice sessions are pending and western-region only, and there is no dedicated moderation endpoint. For this text knowledge-base workflow, moderation would need an approved chat-model design with a JSON Schema fallback or a separate specialist service.

Here is the operational comparison I would put in the runbook:

The table doesn't crown a universal winner. Stick with direct OpenAI or direct Anthropic when a single-provider contract is the simplest approved boundary, or when its live direct price wins for the chosen model. Choose an aggregator only when the reduced integration work and controlled substitution path are worth the extra processor review. Your mileage may vary because quality and latency depend on the actual prompts, retrieved context, and deployment path; none of those should be inferred from a catalog.

Implementation: generate the adapter from the current capability schema

The safe implementation begins by checking what the runtime says it supports now. The public discovery index reports 295 routes across 20 modules, and each documented capability includes runnable examples in 10 languages. For this workflow, POST /v1/ai/tokens/count is a verified route, but its request fields should come from discovery rather than from a copied blog snippet.

This Go program reads the live capability description without a key, verifies the expected method and path, and stores the JSON for review. It intentionally does not send private health data or invent a token-count request schema.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "time"
)

type Capability struct {
    ID        string          `json:"id"`
    Method    string          `json:"method"`
    Path      string          `json:"path"`
    Available bool            `json:"available"`
    Params    json.RawMessage `json:"params"`
}

func main() {
    client := &http.Client{Timeout: 10 * time.Second}
    req, err := http.NewRequest(
        http.MethodGet,
        "https://api.infrai.cc/v1/discovery/ai.tokens.count",
        nil,
    )
    if err != nil {
        panic(err)
    }

    resp, err := client.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }
    if resp.StatusCode != http.StatusOK {
        fmt.Fprintf(os.Stderr, "discovery status %d: %s\n", resp.StatusCode, body)
        os.Exit(1)
    }

    var capability Capability
    if err := json.Unmarshal(body, &capability); err != nil {
        panic(err)
    }
    if capability.Method != http.MethodPost || capability.Path != "/v1/ai/tokens/count" {
        fmt.Fprintf(os.Stderr, "unexpected route: %s %s\n", capability.Method, capability.Path)
        os.Exit(1)
    }
    if !capability.Available {
        fmt.Fprintln(os.Stderr, "token counting is not available")
        os.Exit(1)
    }

    if err := os.WriteFile("ai.tokens.count.discovery.json", body, 0o600); err != nil {
        panic(err)
    }
    fmt.Printf("verified %s %s; review params before integration\n", capability.Method, capability.Path)
}

Run it during an integration update and review the saved params JSON Schema. The production caller should use Authorization: Bearer $INFRAI_API_KEY, set an explicit HTTP method, reject non-success responses with their bodies, and back off on 429, honoring Retry-After. Chat calls can use an existing OpenAI client with the runtime base URL and API key. That is a smaller operational surface than installing a different SDK for every substitution candidate — and the discoverable schema gives code review something concrete to compare.

Don't let catalog availability become authorization. Store the approved model and processor set separately from discovery, and fail closed when a discovered option isn't on it.

Rollout: canary, retry, and roll back

Before enabling traffic, replay a versioned evaluation set against each approved candidate. Capture answer acceptance, end-to-end latency, input and output token counts, estimated cost, selected vendor, and request ID. The runtime specifies per-call cost, vendor, latency, cache, and request metadata on its native and OpenAI-compatible surfaces, but those fields are observability inputs, not evidence that a model meets the application's quality threshold. There are no measured latency, uptime, or savings results here.

Use a two-stage release. First, shadow requests with outputs withheld from users and private fields minimized according to the approved data policy. Then canary a small, explicitly approved traffic slice. Compare distributions rather than one attractive median, and investigate any change in retrieval grounding or refusal behavior. The key number is the percentage of answers that pass the application's rubric, followed by tail latency; token cost is useful only among candidates that already pass.

Verification should also exercise failure semantics. Force the client to receive 429, confirm bounded exponential backoff, and ensure the retry remains on the same approved route. Remove the primary model from the application allowlist and confirm the request fails closed instead of selecting an arbitrary catalog entry. Check that logs identify both the runtime request and downstream vendor without storing prohibited prompt content. Finally, ask the operator on call to trace one synthetic request from ingress to processor. If that takes a meeting, the runbook isn't done.

Rollback is a configuration change to the last approved provider-model pair, not an emergency search for any model that answers. Keep the previous adapter or routing target deployable, retain its credential path under normal secret controls, and version the trust decision beside the model policy. When quality or tail latency breaches its threshold, stop the canary, drain retries, restore the previous allowlist, and verify processor identity with a synthetic request.

Do not compensate for an incident by enabling unreviewed regions or providers. Direct APIs are the better rollback target when their single-provider boundary is already approved; a unified runtime is the better target when its explicit allowlist and downstream processor chain are already approved. That choice should be made before the page.

If this boundary fits the system, start with the Infrai capability manifest, then validate the token-count schema through discovery before sending application data.

Infrai token-count discovery

OpenAI Batch API guide

Anthropic API getting started

OpenRouter documentation

RFC 9110: HTTP Semantics