多 Agent 系统调试：你的 Trace Tree 在说谎

Why

When you deploy an agent-based system to production, you discover something uncomfortable: your system works correctly 99% of the time, but when it fails, it fails in ways you don't expect. Not because of bugs in your business logic—but because of control-flow regressions where an agent starts routing to the wrong tool, or tool-safety regressions where a tool returns an unexpected schema that your agent doesn't handle.

Traditional debugging fails here. Logs give you the inputs and outputs, but the trace tree—the actual decision path your agent took—is opaque. You can't set a breakpoint in a live LLM call. And by the time you reproduce the issue locally, the LLM provider may have updated their model, making exact reproduction impossible.

This is the problem Chronicle solves.

Core Concepts

Decision Graphs

Multi-agent systems aren't linear pipelines. They're decision graphs—directed acyclic graphs where each node represents a decision point (an LLM call), and each edge represents a routing choice (tool calls, conditional branches). When a failure occurs, the question isn't "what went wrong?"—it's "which path through the graph led to the failure?"

Recording

Chronicle intercepts the execution flow at each decision point. For each LLM call, it records:

• The model name and parameters
• The full prompt (including system prompt, conversation history, and tool definitions)
• The raw tool calls and their results

from agent_chronicle import record

with record() as ctx:
    response = agent.complete("Summarize the latest security advisories")
    # Chronicle records: model, params, prompt, tool_calls, results

Cut-Point Replay

Once a failure is recorded, you can replay it deterministically by setting cut points—specific decision points where you want to inject a modified response instead of re-playing the original LLM call:

from agent_chronicle import CutPoint, replay

# Replay the recorded session, but use a fixed response at decision point 3
with replay(ctx.session_id) as session:
    session.cut(
        at_decision=3,
        using=FixedResponse("Here is a safe summary...")
    )
    result = session.resume()

Cut points let you test "what if the LLM had returned X instead of Y?"—without needing live LLM calls.

Quick Start

Installation

pip install agent-chronicle

Record a Session

from agent_chronicle import AgentChronicle

chronicle = AgentChronicle(
    project="my-agent",
    api_key=os.environ["CHRONICLE_API_KEY"]
)

# Wrap your agent
agent = chronicle.wrap(my_agent)

# Run normally—Chronicle records in background
result = agent.run("Query the database for active users")

Create a Regression Test

# From a recorded session, generate a regression test
test = chronicle.generate_test(
    session_id="session_abc123",
    name="test_active_users_query"
)

# Save as a committed test file
test.save("tests/regressions/test_active_users_query.py")

Run Without Live LLM Calls

from agent_chronicle import replay

# Replay a recorded failure
with replay("session_abc123") as session:
    # Apply a fix at decision point 2
    session.cut(at_decision=2, using=MockResponse(...))
    
    # Verify the fix works
    result = session.resume()
    assert result.success

Verification

Chronicle's regression tests are deterministic—they replay the exact same prompt/response pairs every time, without calling the LLM. This means:

1. Fast feedback: No API latency or cost
2. Reproducible: Same result every run
3. Shareable: Commit the test and anyone can reproduce the failure

def test_database_tool_timeout_handling():
    """Regression test: agent should retry on tool timeout."""
    with replay("session_timeout_001") as session:
        # Simulate a timeout at decision point 4
        session.cut(
            at_decision=4,
            using=ToolError("Connection timeout after 30s")
        )
        
        result = session.resume()
        
        # Agent should have retried, not failed
        assert result.tool_calls[-1].name == "retry"
        assert result.success

Compare Runs

Compare two recorded sessions to understand what changed:

diff = chronicle.compare(
    baseline="session_old_001",
    candidate="session_new_001"
)

# See which decision points differ
for decision in diff.changed_decisions:
    print(f"Decision {decision.id}:")
    print(f"  Old: {decision.old_response[:100]}...")
    print(f"  New: {decision.new_response[:100]}...")

Recording Architecture

Chronicle uses a layered recording architecture:

┌─────────────────────────────────────────┐
│           Application Layer             │
│  (your agent code, tool definitions)    │
├─────────────────────────────────────────┤
│           Chronicle Core                 │
│  ┌──────────┐  ┌──────────────────┐    │
│  │ Recorder │  │ Decision Graph    │    │
│  │          │  │ Builder           │    │
│  └──────────┘  └──────────────────┘    │
│  ┌──────────┐  ┌──────────────────┐    │
│  │ Cut-Point│  │ Session Store     │    │
│  │ Engine   │  │ (SQLite/Postgres)│    │
│  └──────────┘  └──────────────────┘    │
├─────────────────────────────────────────┤
│           LLM Provider                  │
│  (OpenAI, Anthropic, local models)     │
└─────────────────────────────────────────┘

The Recorder intercepts all LLM calls and tool executions. The Decision Graph Builder reconstructs the execution path from these recordings. The Cut-Point Engine manages replay with injected responses.

Session Storage

Sessions are stored in a local SQLite database by default, or you can use PostgreSQL for production:

chronicle = AgentChronicle(
    project="my-agent",
    storage=StorageBackend.postgres(
        connection_string=os.environ["DATABASE_URL"]
    )
)

Each session captures:

• Decision graph (nodes and edges)
• Full prompt/response pairs
• Tool call inputs and outputs
• Timing information
• Custom metadata

Cut-Point Replay Deep Dive

Cut points are the key to turning recorded failures into actionable tests. A cut point can inject:

Fixed Responses

session.cut(at_decision=3, using=FixedResponse("Fixed output"))

Mock Tool Results

session.cut(
    at_decision=2,
    using=ToolResult("query_users", {"users": []})
)

Error Injection

session.cut(
    at_decision=4,
    using=ToolError("Simulated connection refused")
)

Partial Modification

# Only modify the tool_calls, keep other fields
session.cut(
    at_decision=3,
    using=PartialResponse(
        tool_calls=[{"name": "safe_tool", "arguments": {}}]
    )
)

Use Cases

Tool-Safety Regressions

When a tool's schema changes (e.g., a field is renamed or removed), Chronicle replays the recorded session and cuts at the decision point to test how your agent handles the new schema:

# Tool schema changed from {user} to {users}
session.cut(
    at_decision=2,
    using=ToolResult(
        "query_users",
        {"users": [{"id": 1, "name": "Alice"}]}  # New schema
    )
)
result = session.resume()
assert result.success  # Agent handled the schema change

Routing Regressions

When routing logic changes and breaks existing paths:

# Simulate a new routing condition
session.cut(
    at_decision=1,
    using=RoutingDecision(next_node="secure_branch")
)
result = session.resume()

Context Window Overflows

Replay with truncated context to test graceful degradation:

session.cut(
    at_decision=5,
    using=ContextTruncation(max_tokens=2048)
)

FAQ

Q: Does Chronicle work with any LLM provider? A: Yes. Chronicle intercepts at the application layer, so it works with OpenAI, Anthropic, local models, and custom LLM backends.

Q: How is this different from just logging LLM calls? A: Traditional logging gives you inputs and outputs. Chronicle records the full decision graph, enabling cut-point replay and deterministic testing.

Q: Can I use Chronicle with existing test suites? A: Yes. Chronicle generates standard pytest-compatible test files that integrate with your existing CI/CD pipeline.

Q: Is there a size limit on recorded sessions? A: Sessions are stored in your own database (SQLite or PostgreSQL), so storage limits depend on your infrastructure.

Roadmap

• [ ] v0.3: Multi-agent session correlation (trace across agent boundaries)
• [ ] v0.4: Visual decision graph debugger
• [ ] v0.5: Cloud replay service for team collaboration
• [ ] v0.6: Integration with existing observability platforms (Datadog, New Relic)

Get Started

pip install agent-chronicle

Then follow the step-by-step onboarding guide to record your first session and create a committed regression test.

---

Chronicle is MIT-licensed and maintained by The Agent Plane. Contributions welcome.