AWS Bedrock 提示缓存实战：输入 Token 成本最高降 90%

Amazon Bedrock 提示缓存可通过重复发送相同上下文来将输入 token 成本降低高达 90%，具体费率见 Amazon Bedrock 提示缓存定价。若不使用缓存，向模型发送一份 10,000 token 的合同并附带 50 个用户问题，意味着将为模型已处理过的内容支付 500,000 个输入 token 的全额费用。

你可以通过缩短提示、缩小上下文窗口或实现应用层缓存来缓解这一问题。但每种方案都涉及权衡：

缩短提示会减少 token 数量，但同时也可能降低上下文质量。

更小的上下文窗口会降低成本，但代价是模型在完整信息上进行推理的能力。

响应缓存在处理相同查询时表现良好，但当相同上下文与不同问题搭配时则毫无收益。

Amazon Bedrock 中的提示缓存在基础设施层面帮助降低这一挑战。当你会话上下文的某些部分（系统提示、文档、工具定义）被缓存后，Amazon Bedrock 在后续请求中直接读取缓存的 token，而无需重新处理。这可以减少首个 token 的生成时间（TTFT），并将缓存命中的输入 token 成本降低高达 90%，同时无需更改模型或降低提示质量。

本文将通过 Amazon Bedrock Converse API 逐步介绍六个实用的提示缓存场景，从基础到高级模式：

消息内容缓存：为多问题分析缓存长文档。

系统提示缓存：在不同会话间缓存角色定义和指令。

工具定义缓存：为智能体工作流缓存工具 schema。

混合 TTL 缓存：为不同内容层级分配不同缓存生命周期。

租户隔离：在多租户应用中实现按租户的缓存分离。

LangChain 集成：在 LangChain 框架中使用提示缓存。

提示缓存工作原理

提示缓存存储部分处理输入的快照，使后续具有相同前缀的请求跳过冗余计算。本节涵盖请求流程、支持模型和定价。

当你在请求中包含 cachePoint 标记时，Amazon Bedrock 会评估该标记前的内容是否与现有缓存条目匹配。如果匹配（缓存命中），模型可以跳过重新处理这些 token，从缓存状态开始生成。如果不存在匹配（缓存未命中），模型处理完整内容并将结果写入缓存，以供后续请求使用。

下图展示了这个流程：

图 1：提示缓存请求流程，其中第一个请求写入缓存，第二个请求从缓存读取，从而降低 TTFT 和输入 token 成本

理解这个流程后，四个关键概念决定了缓存在实际使用中的行为：

缓存作用域：缓存条目按单个 AWS 账户和 AWS 区域作用域限定。

Token 阈值：每个缓存检查点必须满足最低 token 阈值才能激活。例如，Anthropic Claude Sonnet 4.5 和 Sonnet 4.6 每个检查点至少需要 1,024 个 token，而 Opus 模型至少需要 4,096 个。

生存时间（TTL）：缓存条目根据请求中指定的 TTL 过期。默认值为 5 分钟，部分模型最长支持 1 小时。

模型无关语法：Converse API 的 cachePoint 语法在各支持的模型系列中保持一致，包括 Anthropic Claude 和 Amazon Nova。

最新模型支持信息请参阅 Amazon Bedrock 提示缓存文档。

提示缓存在标准输入和输出 token 之外引入了两个 token 类别：

对于具有重复上下文的工作负载，输入 token 成本节省约达 75%。例如，如果你发送一份 10,000 token 的文档并附上 10 个不同的问题，第一个请求会产生缓存写入成本。其余九个请求各自从缓存读取，享受 90% 的成本降低，最终该文档上下文在输入 token 成本上实现约 75% 的净节省。这假设所有后续请求均在 TTL 窗口内完成。过期后的请求会触发新的缓存写入，降低净节省。具体定价信息请参阅 Amazon Bedrock 定价。

在开始使用这些场景前，请确保你具备以下条件：

一个 AWS 账户，在支持的 AWS 区域（如 us-west-2）中拥有 Amazon Bedrock 访问权限。

为目标模型启用了模型访问权限。本文中示例使用 Anthropic Claude Sonnet 4.5（global.anthropic.claude-sonnet-4-5-20250929-v1:0）。有关说明请参阅管理模型访问权限。关于最新的模型和区域可用性，请参阅 Amazon Bedrock 中按 AWS 区域划分的支持模型。这是一个跨区域推理配置文件。请求会自动跨区域路由，这偶尔可能增加缓存写入频率。

Python 3.10 或更高版本，已安装以下依赖：pip install boto3>=1.43.0 langchain-aws>=0.2.12 matplotlib pandas

pip install boto3>=1.43.0 langchain-aws>=0.2.12 matplotlib pandas

注意：使用场景 4（混合 TTL）中 cachePoint 的 ttl 参数需要 Boto3 1.43.0 或更高版本。

AWS 凭证通过默认配置文件或环境变量配置。请参阅配置 AWS 命令行界面（AWS CLI）获取设置说明。

场景 1：消息内容缓存

提示缓存的一个常见用例是缓存你重复查询的长文档或参考内容。例如，在检索增强生成（RAG）应用中，你对同一文档提出多个问题，或者一个编码助手引用一个大型代码库。

在这个场景中，你在静态文档和动态问题之间放置一个 cachePoint 标记。Amazon Bedrock 在第一次调用时缓存文档，并在后续调用时可能复用它。

消息内容缓存工作原理

在静态内容和动态问题之间放置一个 cachePoint 内容块。Amazon Bedrock 缓存检查点之前的所有内容，并在后续请求中复用：

content = [
    {"text": "<static document content>"},
    {"cachePoint": {"type": "default"}},   # cache everything above
    {"text": "<user question>"}             # dynamic, changes per request
]

以下代码通过一个完整的可工作示例来实践这个模式。

首先，设置 Amazon Bedrock 运行时客户端并定义一个示例文档。在生产应用中，这个文档可以是 PDF、知识库文章或其他超过 1,024 token 阈值的内容：

import boto3
import json
import time

MODEL_ID = "global.anthropic.claude-sonnet-4-5-20250929-v1:0"  # Minimum token requirement to activate cache: 1,024 tokens
AWS_REGION = "us-west-2"

bedrock = boto3.client("bedrock-runtime", region_name=AWS_REGION)

# Document LARGE enough (>1024 tokens required for Claude Sonnet 4.5)
DOCUMENT = """
The universe is a vast and mysterious expanse that has captivated human imagination for millennia. From the earliest civilizations who looked up at the night sky and wondered about the nature of the stars, to modern astronomers using sophisticated telescopes and spacecraft to explore distant galaxies, our quest to understand the cosmos continues unabated. This comprehensive overview covers the major components of our universe, from the smallest asteroids to the largest galaxy clusters.

Our solar system, located in the Milky Way galaxy, is home to eight planets, numerous dwarf planets, and countless smaller objects including asteroids, comets, and meteoroids. The Sun, a middle-aged G-type main-sequence star, provides the energy that sustains life on Earth and influences the dynamics of all objects within its gravitational reach.

Mercury, the innermost planet, experiences extreme temperature variations due to its proximity to the Sun and lack of substantial atmosphere. Venus, often called Earth's twin due to its similar size, has a thick atmosphere composed primarily of carbon dioxide, creating a runaway greenhouse effect that makes it the hottest planet in our solar system. Earth, our home, is the only known planet to harbor life, with its unique combination of liquid water, moderate temperatures, and protective magnetic field.

Mars, the Red Planet, has long been a subject of fascination and speculation about the possibility of extraterrestrial life. Its rusty appearance comes from iron oxide prevalent on its surface. The planet features the largest volcano in the solar system, Olympus Mons, and a canyon system, Valles Marineris, that dwarfs the Grand Canyon. Recent Mars missions have discovered evidence of ancient river systems and the presence of water ice beneath the surface.

接下来，定义缓存函数。核心要素是将 `cachePoint` 块放在静态文档和动态问题之间：

```python
def converse_with_cache(document, question):
    """Query a document with prompt caching enabled."""
    content = [
        {"text": document},
        {"cachePoint": {"type": "default"}},
        {"text": question}
    ]

response = bedrock.converse(
        modelId=MODEL_ID,
        messages=[{"role": "user", "content": content}],
        inferenceConfig={"maxTokens": 512}
    )
    return response["usage"]

运行两次请求来观察缓存行为。第一次调用填充缓存，后续调用使用不同问题则复用缓存：

# Request 1: cache write (first time seeing this content)
usage1 = converse_with_cache(DOCUMENT, "What are the key points?")
print("Request 1 (cache write expected):")
print(json.dumps(usage1, indent=2))

time.sleep(1)

# Request 2: cache read (same document, different question)
usage2 = converse_with_cache(DOCUMENT, "Summarize the conclusions.")
print("\nRequest 2 (cache read expected):")
print(json.dumps(usage2, indent=2))

读取缓存指标

响应中的 usage 对象包含两个与缓存相关的字段：

在我们使用 Anthropic Claude Sonnet 4.5 对超过 1,024 个 token 的文档进行测试时，首次响应的缓存写入如下：

{
  "inputTokens": 28,
  "outputTokens": 253,
  "cacheWriteInputTokens": 1898,
  "cacheReadInputTokens": 0
}

使用相同文档前缀的后续请求产生了缓存读取：

{
  "inputTokens": 28,
  "outputTokens": 294,
  "cacheWriteInputTokens": 0,
  "cacheReadInputTokens": 1898
}

注意，cacheReadInputTokens 现在反映了从缓存中读取的 1,898 个 token。整个文档前缀被复用，无需重新处理。只有 28 个 token（问题本身）作为标准输入被处理。这些缓存的 token 按降低的缓存读取费率计费（比标准输入低 90%）。

简化的缓存管理

Amazon Bedrock 上的 Claude 模型支持简化的缓存管理。你只需放置一个 cachePoint，Amazon Bedrock 会自动检查该标记前最多约 20 个内容块的前缀是否命中缓存。你不需要手动放置多个缓存检查点来获取更早对话部分的缓存命中。

如需更细粒度的控制，你可以在每个内容段落后放置多个 cachePoint 标记：

content = [
    {"text": SECTION_1},
    {"cachePoint": {"type": "default"}},
    {"text": SECTION_2},
    {"cachePoint": {"type": "default"}},
    {"text": SECTION_3},
    {"cachePoint": {"type": "default"}},
    {"text": QUESTION}
]

这种方法支持部分缓存命中。如果只有前两个段与之前的请求匹配，模型会复用这些段的缓存，并处理剩余内容。

相同的缓存语法也适用于 converse_stream。主要区别在于缓存指标在流末尾的 metadata 事件中到达，而不是在即时响应中：

def converse_stream_with_cache(document, question):
    content = [
        {"text": document},
        {"cachePoint": {"type": "default"}},
        {"text": question}
    ]

response = bedrock.converse_stream(
        modelId=MODEL_ID,
        messages=[{"role": "user", "content": content}],
        inferenceConfig={"maxTokens": 512}
    )

text = ""
    usage = {}
    for event in response["stream"]:
        if "contentBlockDelta" in event:
            text += event["contentBlockDelta"]["delta"].get("text", "")
        elif "metadata" in event:
            usage = event["metadata"].get("usage", {})

return usage, text

为了量化延迟改进，你可以测量有缓存和无缓存情况下的 TTFT：

def measure_ttft(document, question, use_cache=True, iterations=5):
    """Measure TTFT using ConverseStream."""
    ttfts = []
    for i in range(iterations):
        if use_cache:
            content = [
                {"text": document},
                {"cachePoint": {"type": "default"}},
                {"text": question}
            ]
        else:
            content = [{"text": document + "\n\n" + question}]

start = time.time()
        response = bedrock.converse_stream(
            modelId=MODEL_ID,
            messages=[{"role": "user", "content": content}],
            inferenceConfig={"maxTokens": 512}
        )
        for event in response["stream"]:
            if "contentBlockDelta" in event:
                ttft = time.time() - start
                ttfts.append(ttft)
                for _ in response["stream"]:
                    pass
                break
        time.sleep(0.5)
    return ttfts

提示缓存可以降低 TTFT，且收益随缓存前缀大小的增加而增长。对于较小的文档（约 2,000–5,000 个 token），在小样本迭代下改进可能不具有统计显著性。具体的改进幅度因文档大小、模型和当前负载而异。对于较大的缓存前缀（超过 10,000 个 token），TTFT 的降低会更加明显。

场景 2：系统提示缓存

许多应用使用详细系统提示来定义模型的人设、指南和领域专业知识。这些系统提示可能跨越数千个 token，并在用户交互中保持不变。通过系统提示缓存，你只需支付一次完整处理成本，之后每条消息都可以复用缓存的系统提示。

缓存点放在 system 参数内部，与用户消息分开。

Converse API 的 system 参数接受内容块数组。在系统文本后放置一个 cachePoint：

system = [
    {"text": "<detailed system prompt>"},
    {"cachePoint": {"type": "default"}}
]

下一个示例展示如何在完整请求中使用详细人设提示。

以下示例定义了一个全面的系统提示：一个专业太空科学顾问人设，包含详细的响应指南。该提示超过了缓存所需的 2,048 个 token 阈值：

SYSTEM_PROMPT = """You are an Expert Space Science Advisor, a highly knowledgeable
AI assistant specializing in astronomy, astrophysics, planetary science, and space
exploration. Your role is to provide accurate, comprehensive, and engaging information
about many aspects of space science.

## Core Expertise Areas

### Planetary Science
You possess deep knowledge of planetary formation, composition, atmospheres, and
geology across our solar system and beyond. This includes understanding the inner
rocky planets (Mercury, Venus, Earth, and Mars), gas giants (Jupiter and Saturn),
ice giants (Uranus and Neptune), dwarf planets such as Pluto and Ceres, and the
countless smaller bodies that inhabit our solar system.

You understand the processes that shape planetary worlds: volcanism, tectonics,
atmospheric erosion, meteorite impacts, cryovolcanism, and planetary differentiation.
You can explain how planetary magnetic fields arise and protect atmospheres, how
seasons change on different worlds, and how water and other volatiles behave under
varying planetary conditions.

Your expertise extends to moons and satellite systems. You know the fascinating
moons of the solar system: Jupiter's volcanic Io, Europa with its subsurface ocean,
Saturn's Titan with its methane lakes, Enceladus with its water geysers, Neptune's
Triton with its frozen nitrogen, and our own Moon with its unique formation history.
You understand how tidal forces shape these worlds and how they may harbor habitable
environments.

### Astrophysics and Cosmology
Your expertise extends to the fundamental physics governing the universe. You
understand stellar evolution from star-forming nebulae to supernovae and black holes.
You can explain the life cycle of stars, from long-lived red dwarfs to short-lived
blue giants, and how stellar mass determines a star's fate.

You comprehend the structure and evolution of galaxies, including our Milky Way.
You know the different types of galaxies (spirals, ellipticals, and irregulars)
and how they form, interact, and evolve over billions of years. You understand the
role of supermassive black holes at galactic centers and how they influence galactic
evolution.

Your knowledge encompasses modern cosmology: the Big Bang theory, the expansion of
the universe, the cosmic microwave background radiation, dark energy, dark matter,
and the large-scale structure of the cosmos. You can discuss the ultimate fate of
the universe, the nature of spacetime, and the fundamental principles governing
physical reality.

### Space Exploration
You are well-versed in the history and future of human and robotic space exploration.
You know the milestones of the space age: the launch of Sputnik, Yuri Gagarin as
the first human in space, the Apollo program and Moon landings, the Skylab, Mir,
and ISS space stations, the space shuttle programs, and the pioneering robotic
missions.

You understand current and past robotic missions: the Voyager probes exploring
interstellar space, Cassini-Huygens at Saturn, New Horizons at Pluto, the Mars
rovers Spirit, Opportunity, Curiosity, and Perseverance, ESA missions like Rosetta,
Chinese lunar missions, and the Parker Solar Probe and Solar Orbiter.

You are familiar with space technology: propulsion systems (chemical, ionic,
nuclear), space navigation, life support systems, radiation protection, deep-space
communications, space telescopes, and scientific instruments. You understand the
unique engineering challenges of spaceflight: vacuum, radiation, microgravity,
extreme temperatures, and long-term reliability.

You follow current developments in commercial space exploration: SpaceX and its
Starship system, Blue Origin and New Glenn, NASA's Artemis plans to return to the
Moon, the lunar Gateway space station, planned missions to Mars, and the growing
role of private companies in space exploration.

### Astrobiology and the Search for Life
You have specialized knowledge about the conditions necessary for life, habitable
zo

缓存函数将 cachePoint 放在 system 参数中，将用户消息隔离开来：

```python
def converse_system_cached(question):
    response = bedrock.converse(
        modelId=MODEL_ID,
        system=[
            {"text": SYSTEM_PROMPT},
            {"cachePoint": {"type": "default"}}
        ],
        messages=[{"role": "user", "content": [{"text": question}]}],
        inferenceConfig={"maxTokens": 512}
    )
    return response["usage"], response["output"]["message"]["content"][0]["text"]

# Request 1: cache write
usage1, _ = converse_system_cached("What are the most promising locations for finding microbial life?")
print("Request 1 (cache write expected):")
print(json.dumps(usage1, indent=2))

time.sleep(1)

# Request 2: different question, same system prompt (cache read)
usage2, _ = converse_system_cached("How do gravitational waves help us study the universe?")
print("\nRequest 2 (cache read expected, different user question):")
print(json.dumps(usage2, indent=2))

用户消息在请求之间发生变化，但 system prompt 保持不变。Amazon Bedrock 设计为缓存 system prompt 前缀并重复使用它，无论用户询问什么内容。

何时使用 system prompt 缓存

System prompt 缓存适用于以下场景：

• 具有详细角色描述和行为准则的基于人格的 AI 智能体。
• 具有广泛指令且在多轮交互中保持不变的 Agentic 工作流。
• 具有复杂公司政策和响应协议的客服机器人。
• 在 system prompt 中嵌入知识库的领域特定 AI 智能体。

场景 3：工具定义缓存

Agentic 应用程序通常定义数十个工具，每个工具都具有详尽的 JSON schema。这些工具定义总共可能包含数千个 token，且在请求之间几乎不会变化。工具定义缓存可防止在每次交互中重新处理这些 schema。

将 cachePoint 作为 tools 数组中的最后一个元素追加到 toolConfig 中：

toolConfig = {
    "tools": [
        {"toolSpec": {"name": "tool_1", "description": "...", "inputSchema": {"json": {...}}}},
        {"toolSpec": {"name": "tool_2", "description": "...", "inputSchema": {"json": {...}}}},
        {"cachePoint": {"type": "default"}}   # after the tool definitions
    ]
}

以下示例演示了一组collectively exceed the token threshold 的空间主题工具 schema。在生产应用中，这些可能是 API 集成、数据库查询或外部服务调用：

TOOLS_LIST = [
    {
        "name": "analyze_celestial_object",
        "description": "Analyzes a celestial object and returns detailed information about its physical properties, composition, orbit, and observational characteristics. Supports planets, moons, stars, exoplanets, and galaxies with different levels of analysis depth.",
        "input_schema": {
            "type": "object",
            "properties": {
                "object_name": {
                    "type": "string",
                    "description": "The name of the celestial object to analyze (e.g., Mars, Titan, Pr