中国廉价LLM API背后的GPU数学：为何$0.35/M成为可能

Two months ago I ran a comparison that made me double-check my receipt: 50,000 API calls on DeepSeek V4 Flash cost about as much as a single meal, while the same traffic on GPT-4o was four figures. My first reaction wasn't "great deal" — it was "what's the catch?" Everyone assumed the cheap Chinese models were subsidized or selling at a loss to buy market share. So I did the boring thing and tried to derive the cost from first principles: what does it actually cost to serve one of these models on the hardware these labs can actually buy? The answer surprised me. The prices aren't a subsidy. They're the memory-bandwidth floor, computed in public.

The hardware constraint nobody talks about

Here's the part that doesn't show up in benchmark charts: Chinese labs can't buy the GPUs that American labs use. Export controls keep the newest NVIDIA parts out of the country, so the best available hardware is roughly the H800 (the NVLink-capped H100 variant), the H20, and domestic accelerators like Huawei's Ascend line. That's not a footnote — it's the entire business model.

Scarcity is a brutal optimizer. You can't out-scale your way to better inference economics with a few thousand of the latest GPUs, so you have to out-engineer. The result is an efficiency stack so aggressive that it changes the unit economics of serving: sparse MoE architectures (I covered the architecture side in my earlier post), FP8/INT4 weight quantization, and ruthless continuous batching.

The binding constraint is memory, not compute

Here's the mental model that changed how I read inference pricing. During generation, every output token requires reading the model's active weights from HBM once. For a dense 235B model in FP8 that's 235 GB of reads per token. An H800 delivers ~3.35 TB/s of memory bandwidth — so the ceiling is roughly 14 tokens/s. Fourteen. That's not a product, that's a screensaver.

MoE fixes this because only the active experts get read. A 235B-parameter model with 22B active parameters (DeepSeek V4 Flash and Qwen3-235B-A22B are both built this way) reads ~22 GB per token — a 10x jump to ~150 tokens/s ceiling before quantization. Then you quantize the weights to INT4/FP8, cutting the bytes per parameter in half:

# Serve a Qwen3-235B-A22B-class MoE on one H800 with quantized weights
vllm serve Qwen/Qwen3-235B-A22B \
  --quantization fp8 \
  --max-model-len 32768 \
  --max-num-seqs 256 \
  --gpu-memory-utilization 0.92 \
  --enable-prefix-caching

Now the active weights are ~11 GB, the ceiling is ~300 tokens/s, and the remaining gap to real throughput is filled by continuous batching — multiplexing dozens of concurrent streams per GPU so the memory bus never goes idle.

Deriving the floor: what one GPU-hour must earn

With those pieces in place, the floor is just arithmetic. I'll use public assumptions and let you change any of them:

GPU_COST = 25_000         # $ H800-class street price in China (reported range ~$20-40K)
GPU_LIFETIME_YEARS = 4    # typical data-center depreciation
HOURS_PER_YEAR = 8_760

gpu_per_hour = GPU_COST / (GPU_LIFETIME_YEARS * HOURS_PER_YEAR)

ACTIVE_PARAMS_B = 22      # MoE active parameters per token
BYTES_PER_PARAM = 0.5     # INT4/FP8 mixed quantization
MEM_BW_GBPS = 3_350       # H800 HBM3 bandwidth (GB/s)

weights_gb = ACTIVE_PARAMS_B * BYTES_PER_PARAM      # 11 GB read per token
ceiling_tps = MEM_BW_GBPS / weights_gb              # memory-bound ceiling
realistic_tps = ceiling_tps * 0.6                   # ~60% batching efficiency

tokens_per_hour = realistic_tps * 3_600
floor_per_1m = gpu_per_hour / tokens_per_hour * 1_000_000

print(f"GPU cost: ${gpu_per_hour:.2f}/hr")
print(f"Memory-bound ceiling: {ceiling_tps:.0f} tok/s")
print(f"Realistic sustained:  {realistic_tps:.0f} tok/s")
print(f"Hardware floor: ${floor_per_1m:.2f} per 1M output tokens")

GPU cost: $0.71/hr
Memory-bound ceiling: 304 tok/s
Realistic sustained:  183 tok/s
Hardware floor: $1.08 per 1M output tokens

Now check the sticker price: DeepSeek V4 Flash charges $1.10 per 1M output tokens. The list price is within two cents of the hardware floor. That's the whole story — the lab isn't bleeding money to buy adoption, and it isn't charging a 50x margin either. It's charging memory-bandwidth cost plus a sliver, and making it work through scale and software. The input price of $0.35/M is the same physics from the other side: prefill is compute-bound and parallelizable, so tokens are cheaper to produce, which is why input costs a third of output.

What that means for your bill

Here's the practical version, using the canonical prices for the four models I actually use (per 1M tokens):

A small SaaS doing 30,000 requests a month at ~1,680 input + ~190 output tokens per request:

STEPS = 30_000         # requests per month
IN_PER_STEP = 1_680    # avg input tokens per request
OUT_PER_STEP = 190     # avg output tokens per request
FINAL_OUT = 0

def monthly_cost(p_in, p_out):
    total_in = STEPS * IN_PER_STEP
    total_out = STEPS * OUT_PER_STEP + FINAL_OUT
    return (total_in * p_in + total_out * p_out) / 1_000_000

for name, p_in, p_out in [
    ("DeepSeek V4 Flash", 0.35, 1.10),
    ("Qwen3-235B-A22B",   1.60, 6.40),
    ("GLM-5-130B",        1.20, 4.80),
    ("GPT-4o",           10.00, 30.00),
]:
    print(f"{name:18} ${monthly_cost(p_in, p_out):.2f} / month")

DeepSeek V4 Flash   $23.91 / month
Qwen3-235B-A22B     $117.12 / month
GLM-5-130B          $87.84 / month
GPT-4o              $675.00 / month

Same workload, same quality bar for most tasks, 28x difference in the bill. That spread isn't marketing — it's the difference between serving on hardware whose floor is a dollar per million tokens and serving on newer silicon with a thicker margin baked in.

The honest part: where the cheap floor bites

Razor-thin margins are a two-way street, and there are real trade-offs you should price in:

Throughput per stream is modest. My own tests show ~48 tokens/s on DeepSeek V4 Flash vs ~55 on GPT-4o, and GLM-5-130B sits around 31 with occasional mid-stream pauses. The 180 tok/s/GPU number only materializes with many concurrent users — a single chat stream won't see it.

Hard reasoning still favors GPT-4o. In my eval harness, GPT-4o beat DeepSeek on code (76 vs 68 pass@1) and function calling (92 vs 87). Cheap models fail differently — malformed JSON on gnarly schemas, confident nonsense on hard multi-step problems. You need validation and fallbacks, which I wrote up in my model router post.

Capacity is the real risk. A provider pricing at the floor has no slack. When a model goes viral, expect latency spikes and queueing — I've seen it happen more than once. Cheap and infinite are different things.

Prices and models move. The labs republish prices and silently upgrade models. The floor I derived today is for today's silicon and today's quantization tricks.

One API key instead of three dashboards

If the numbers above look appealing, the annoying part is access: DeepSeek, Qwen, and GLM each have their own console, billing, rate limits, and — for many of us — a China phone number requirement at signup. I route everything through tokencnn.com: a single OpenAI-compatible endpoint where deepseek-v4-flash, qwen3-235b-a22b, and glm-5-130b sit behind one API key, so switching models is a one-line change:

curl https://api.tokencnn.com/v1/chat/completions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Explain MoE in one paragraph."}
    ]
  }'

Sign up with just an email — no China phone number, no WeChat — and the $1 free credit covers roughly a month of the workload above.

I stopped assuming cheap Chinese models were subsidized the day I derived $1.08 from a memory-bandwidth equation and found the price at $1.10. The export-controls constraint didn't hurt these labs as much as people expected — it forced them to build the most efficient serving stack in the industry, and that efficiency is exactly what you're buying when you send a request to a $0.35/M API. The catch isn't a loss-leader; it's that the margin is thin, so you're betting on their scale and software staying ahead of the hardware curve.

Have you ever derived the "impossible" price of a product from first principles and found it wasn't impossible at all? I'd love to hear your hardware-floor story.