AI Agent Replay and Tracing Guide

Tracing and replay answer different questions

An AI agent trace is a time-ordered record of model calls, tool calls, MCP operations, agent boundaries, errors, and relevant payloads. It is the evidence for what happened in a particular execution.

Replay uses selected recorded evidence during another invocation. It helps hold part of the execution still while you test a hypothesis. A normal rerun cannot provide the same comparison when model sampling, tool data, network state, or timing all change together.

Capture a baseline that can support replay

Instrument the existing Python runtime at the boundaries that matter. Tool and MCP spans need complete inputs and outputs for exact replay. LLM spans need the request messages and model response if the investigation will hold them recorded or pause at a message or output boundary.

from debrix import (
    SpanKind,
    configure,
    force_flush,
    trace_agent,
    trace_span,
    trace_tool,
)

configure(batch=False, service_name="support-agent")

@trace_tool(name="lookup_account")
def lookup_account(account_id: str) -> dict:
    return support_api.lookup(account_id)

@trace_agent(name="support_agent")
def run_agent(account_id: str, question: str) -> str:
    account = lookup_account(account_id)
    messages = build_messages(account, question)

    with trace_span("complete", kind=SpanKind.LLM) as span:
        span.record_messages(messages)
        answer, usage = call_model(messages)
        span.record_response(dict(content=answer, usage=usage))
        return answer

print(run_agent("acct_42", "Why was my request denied?"))
force_flush()

Debrix stores complete instrumented message and response payloads locally. Bounded previews keep the UI responsive, while the immutable payload evidence supports comparisons and replay decisions.

When a run ends with a wrong answer rather than a thrown error, mark it failed and preserve it as a Failure before the source trace is cleaned up. Add the expected outcome and how it will be evaluated.

Replay through a guided Failure experiment

Choose one or more captured boundaries in the Failure workspace, arm the experiment, and invoke the existing instrumented project. Debrix rebuilds the recorded prefix in order and pauses at each selected boundary. The engineer or coding agent decides whether to keep the recorded value, edit it, or skip it before execution continues.

Read provenance as part of the result

• Recorded: returned from immutable evidence instead of the real dependency.
• Edited: deliberately changed for this experiment.
• Live: executed by the real runtime after the controlled boundary.

Tools-only replay holds supported Tool and MCP outputs recorded. A tools-plus-LLM replay can also return pinned model responses through the public Debrix LLM helper. When a controlled experiment owns a boundary, control loss fails closed instead of silently switching to an unmanaged live call.

Debrix owns the replay and intervention plan, not the application process. The user or coding agent continues to invoke the existing project command; the desktop app does not become a custom agent runtime.

State only what the replay evidence proves

Compare the branch with its original or parent evidence. A single edited boundary can support a bounded causal statement: for example, “downstream behavior succeeds when this recorded tool result is replaced with valid data.” It does not prove why the original tool result was wrong.

Multiple edits form a compound branch. The result describes the combined effect and should not be presented as proof that each edit was necessary or sufficient. Later live calls remain nondeterministic even when the prefix was replayed exactly.

Keep the validator visible

Every conclusion depends on an approved outcome contract: a deterministic test or assertion, a trace or state predicate, a human verdict, or a clearly labelled probabilistic judge. Record pass, fail, or inconclusive together with that source. Without a validator, the replay is inspectable evidence—not proof of success.

Remove replay before verifying the actual fix

Use the replay result to decide what production code, prompt, schema, model, or policy to change. Then start managed verification from the Debrix UI or MCP and run the original scenario through the existing project using the one-time launch environment.

The verification path bypasses supported Tool Mocker, replay, and controlled-branch behavior before those controls can influence the execution. A passing current-revision result can create immutable proof that the real change—not the exploratory branch—fixed the Failure.

Generate a regression from that proof and rerun it through the same no-override path. The original trace, every experiment, the verified change, and later regression results remain linked without rewriting historical evidence.

AI agent replay and tracing checklist

1. Capture complete inputs and outputs at every boundary you may replay.
2. Preserve one representative failed run and define the expected outcome.
3. Choose the smallest intervention that distinguishes the current hypotheses.
4. Inspect recorded, edited, and live provenance before drawing a conclusion.
5. Label multi-edit results as combined effects.
6. Apply the real production change outside the experiment.
7. Verify the original scenario with all exploratory controls bypassed.
8. Generate and rerun a regression from the verified proof.