堆栈跟踪会说谎：OSS Patch 前的边界契约

A first-time contributor cloned a widely used HTTP client and tried to close a stale timeout issue. The issue template pointed at retry.py, and an assistant proposed a backoff rewrite with a passing unit test. The maintainer declined the pull request because idle sockets died in the connection pool, two packages away. The unit test had stubbed the network, so the false diagnosis never had a chance to fail.

This article treats that scene as a composite of common open-source review comments, not as a measured case study. The practical failure is consistent across many first patches: the first named file is treated as the root cause. A cheaper loop writes a boundary contract that fails in production-like conditions before any helper is rewritten. Free-tier models then review the assumption log, instead of inventing a diagnosis from the stack trace alone.

Why the named file is usually a symptom

Open-source issues often include a stack trace, a gist, and a confident comment that names a helper. That bundle is useful evidence, yet it remains a poor specification for a production patch in a busy repository. Retry helpers, formatters, and cache wrappers appear in traces because they sit on every hot path. They show up because they wrap other work, not because they own the failing behavior.

Agentic coding tools make the same move humans make, only faster: they fill missing modules with confident assumptions. The prompt usually contains the issue text, the named file, and a request for a patch, so sibling packages never enter the context window. Timeouts become retry bugs, 404 responses become routing bugs, and flaky tests become missing sleep calls in helpers. None of those guesses is cheap to falsify when the existing suite stubs the real boundary away.

A short decision table before any edit

Use this table before opening an editor, and treat the rows as a proposed filter rather than a measured benchmark. If two or more rows apply, the patch almost certainly spans more than one file. Record that suspicion in the assumption log before a model is invited to write code.

Artifact: an assumption log the model cannot skip

The original artifact in this workflow is a small YAML file that must exist before any model is asked for a diff. The log is committed on the contributor branch as ASSUMPTIONS.yml so reviewers can see every guess. Status values are deliberately boring: untested, supported, or contradicted. A green unit test does not change those values by itself.

# ASSUMPTIONS.yml — proposal template for an OSS fix branch
issue: "https://github.com/example/httpkit/issues/1842"
claimed_file: "httpkit/retry.py"
boundary_under_test: "httpkit.Client.request"
reproduced_on:
  os: "linux"
  runtime: "python3.11"
  package_from: "git+https://github.com/example/httpkit@abc1234"
assumptions:
  - id: A1
    claim: "Timeouts originate in exponential backoff math"
    status: untested
    evidence: "stack frame in retry.py:88"
  - id: A2
    claim: "The connection pool idle timeout is unrelated"
    status: untested
    evidence: "not mentioned in the issue body"
  - id: A3
    claim: "The unit suite's socket stub matches production idle behavior"
    status: contradicted
    evidence: "tests/test_retry.py monkeypatches socket.create_connection"
disallowed_edits:
  - "retry.py until A1 is tested against a live idle socket"
  - "public API signatures"

A model that proposes a diff while any assumption remains untested is still guessing, no matter how tidy the generated helper test looks. Contributors should refuse that diff until the boundary script has run against the reported commit. Reviewers can read the YAML in a minute and see whether the investigation actually moved.

Reproduce at the boundary, not at the helper

The next artifact is a reproduction script that talks to the public surface, not the internal helper. The script below is an example for a fictional httpkit client and should be adapted, not pasted into an unrelated repository. It holds the first byte long enough to expose an idle timeout that retry math cannot see.

# repro_idle_timeout.py — example, not a claim about a real project
import os
import socket
import threading
import time

from httpkit import Client

HOLD_SECONDS = float(os.environ.get("HOLD_SECONDS", "35"))
CLIENT_IDLE = float(os.environ.get("CLIENT_IDLE", "30"))

def slow_server(port_file: str) -> None:
    srv = socket.socket()
    srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    srv.bind(("127.0.0.1", 0))
    srv.listen(1)
    host, port = srv.getsockname()
    with open(port_file, "w", encoding="utf-8") as handle:
        handle.write(str(port))
    conn, _ = srv.accept()
    time.sleep(HOLD_SECONDS)
    conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nOK")
    conn.close()
    srv.close()

def main() -> None:
    port_file = "/tmp/httpkit-repro-port"
    thread = threading.Thread(target=slow_server, args=(port_file,), daemon=True)
    thread.start()
    time.sleep(0.2)
    port = int(open(port_file, encoding="utf-8").read())
    client = Client(base_url=f"http://127.0.0.1:{port}", idle_timeout=CLIENT_IDLE)
    response = client.request("GET", "/")
    assert response.status_code == 200, response.status_code

if __name__ == "__main__":
    main()

Run it in a clean environment so laptop DNS caches and leftover virtualenvs cannot hide the bug. A failing boundary script is the only green light to edit production code. If the script passes on the reported commit, the issue is incomplete, and the next step is a comment on the tracker rather than a pull request.

python -m venv .venv
. .venv/bin/activate
pip install -e ".[test]"
HOLD_SECONDS=35 CLIENT_IDLE=30 python repro_idle_timeout.py
echo $?   # non-zero means the boundary still fails

Turn the reproduction into a contract test

Once the boundary fails, freeze it as a test that maintainers can run without reading the YAML. Prefer the project's existing harness so the contract is not a private ritual on one laptop. The example uses pytest and a local socket, which means CI does not need the public internet.

# tests/test_idle_contract.py — example contract, unexecuted against a real suite
import socket
import threading
import time

from httpkit import Client

def test_idle_timeout_does_not_kill_a_slow_first_byte(tmp_path):
    hold = 35

    def server():
        srv = socket.socket()
        srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        srv.bind(("127.0.0.1", 0))
        srv.listen(1)
        (tmp_path / "port").write_text(str(srv.getsockname()[1]))
        conn, _ = srv.accept()
        time.sleep(hold)
        conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nOK")
        conn.close()
        srv.close()

    threading.Thread(target=server, daemon=True).start()
    time.sleep(0.2)
    port = int((tmp_path / "port").read_text())
    client = Client(base_url=f"http://127.0.0.1:{port}", idle_timeout=30)
    response = client.request("GET", "/")
    assert response.status_code == 200

Commands that belong in the pull request body are ordinary and copy-pasteable. Update ASSUMPTIONS.yml as those commands run, because the log is the investigation record rather than leftover notes. Mark A1 contradicted if changing backoff never flips the contract. Mark A2 supported if changing pool idle behavior does; only then is a patch in pool.py justified.

pytest tests/test_idle_contract.py -vv
git diff --stat
git log --oneline origin/main..HEAD

Gate the branch with a boring checker

A third artifact is a tiny checker that fails the branch when untested rows remain. The script is a proposal and is not tied to a particular CI vendor. Contributors can run it locally, then add it as a pre-push hook if the repository allows extra scripts.

# scripts/check_assumptions.py — proposal script, not a published CI plugin
from __future__ import annotations

import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    sys.stderr.write("PyYAML is required for scripts/check_assumptions.py\n")
    sys.exit(2)

ALLOWED = {"untested", "supported", "contradicted"}

def main(path: str) -> int:
    data = yaml.safe_load(Path(path).read_text(encoding="utf-8"))
    assumptions = data.get("assumptions") or []
    if not assumptions:
        sys.stderr.write("ASSUMPTIONS.yml has no assumptions list\n")
        return 1
    untested = []
    for row in assumptions:
        status = row.get("status")
        ident = row.get("id")
        if status not in ALLOWED:
            sys.stderr.write(f"unknown status {status!r} on {ident}\n")
            return 1
        if status == "untested":
            untested.append(str(ident))
    if untested:
        sys.stderr.write("untested assumptions: " + ", ".join(untested) + "\n")
        return 1
    print("assumption log is complete")
    return 0

if __name__ == "__main__":
    if len(sys.argv) != 2:
        sys.stderr.write("usage: python scripts/check_assumptions.py ASSUMPTIONS.yml\n")
        sys.exit(2)
    sys.exit(main(sys.argv[1]))

pip install pyyaml
python scripts/check_assumptions.py ASSUMPTIONS.yml
# optional local hook
echo 'python scripts/check_assumptions.py ASSUMPTIONS.yml' >> .git/hooks/pre-push
chmod +x .git/hooks/pre-push

This checker does not prove the patch is correct. It only proves the contributor did not skip the log. That is enough to stop the most common drive-by failure: a polished diff attached to an unexamined story about the named helper.

Where a free model is allowed to speak

After the contract fails for the right reason, a free-tier coding model is useful as a reviewer of the log and the diff. It is a poor author of the first diagnosis, because it will reattach the stack trace to the helper. Paste the YAML, the contract test, and git diff, then ask for leftover assumptions, public signature changes, and tests that stub the same boundary.

MonkeyCode is relevant at this review step because the operator-supplied free model access can host the assumption review, and the free server option can run the boundary script away from a developer laptop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The same workflow still holds if another editor and another machine are used; the product is a convenience for the review step, not a substitute for the failing contract.

A prompt that stays inside that scope looks like the following block. It is a proposal for a review message, not a claim about any particular hosted model.

You are reviewing an OSS patch, not proposing a new root cause.
Inputs: ASSUMPTIONS.yml, tests/test_idle_contract.py, and the unified diff.
Return:
- a list of assumption ids still marked untested
- any public API change lacking a changelog entry
- any test that monkeypatches the same boundary as the contract
Do not suggest additional refactors. Do not invent files that are not in the diff.

Keep the model off the first diagnosis on purpose. The free-tier capacity is spent on catching leftover assumptions, not on rewriting helpers that merely appeared in a stack trace. If the model suggests a refactor outside the diff, discard that suggestion and leave the public API alone.

A maintainer-facing pull request skeleton

Reviewers should not have to reverse-engineer the investigation from a wall of comments. A short template reduces drive-by patches that look complete and still miss the contract. The skeleton below is a proposal for the pull request body, not a required house style.

## Contract
- Boundary: `Client.request` against a slow first byte
- Command: `pytest tests/test_idle_contract.py -vv`
- Result on main: fails with idle disconnect
- Result on this branch: passes

## Assumptions
- A1 (retry math) contradicted by the contract
- A2 (pool idle timeout) supported
- A3 (stubbed sockets) contradicted; stubs were not used here

## Out of scope
- No retry API changes
- No dependency bumps
- No formatter-only edits

Numbered hygiene checks before clicking Create pull request:

The contract test is in the first commit, and the fix is in the second.

ASSUMPTIONS.yml has no untested rows left, or each leftover row is explained.

git diff origin/main --stat shows no drive-by refactors or formatter-only noise.

The issue is linked, and the reproduction command is copy-pasteable in the description.

Those four checks are boring on purpose. Maintainers spend their scarce time on ownership questions, not on reconstructing whether the contributor ever left the helper file.

This loop is slower than asking a model to fix the issue number and pasting the first diff into a branch. It also fails when the project has no public boundary to script, such as a GUI with no headless harness. Contract tests that sleep for tens of seconds will annoy CI owners unless they are marked and isolated.

Free model reviews can still miss semantic versioning breaks and missing license headers in new files. They cannot prove that a local socket server matches a vendor load balancer in production. The YAML file is a communication tool for humans, and it is not a formal specification language with tooling guarantees.

Who should not use this approach

Contributors sending one-line documentation or typo fixes do not need an assumption log.

Security patches that require coordinated disclosure should not be developed on a shared free server.

Maintainers of projects without automated tests will not get signal from a contract that cannot run in CI.

Anyone hoping the model will discover the root cause from the issue title alone will be fighting the workflow.

The stack trace remains a useful map of the call chain, and it is a weak map of ownership. Writing the failing contract first keeps both humans and models from renovating the retry helper while the connection pool quietly drops idle sockets. If a free-tier machine is needed to run the boundary script and the assumption review, MonkeyCode's free model access and free server option can host that loop without turning the pull request into a product demo.

For further actions, you may consider blocking this person and/or reporting abuse