Normalize traces
Convert recorded events into one versioned replay format.
AuraOne Open / Agent regression testing
Turn a recorded agent trace into a canonical replay file, assert tool order, retries, and answers, and generate a pytest case that runs offline.
pip install tool-call-replayThe example inputs live in the repository. Get them first:
git clone --depth 1 https://github.com/auraoneai/tool-call-replay.git && cd tool-call-replayThen run the example from the repository root:
tool-call-replay ingest examples/recovered_trace.jsonl --out refund-replay.jsontool-call-replay run refund-replay.json --assert examples/refund_assertions.yaml --report /dev/stdouttool-call-replay pytest refund-replay.json --assert examples/refund_assertions.yaml --out test_refund_replay.py# Tool Call Replay Report
- Trace: `refund-fixed-001`
- Goal: Refund order A100 without issuing duplicate credits.
- Events: `5`
- Assertions: `4`
## Assertions
| Result | Assertion | Detail |
| --- | --- | --- |
| pass | `tool_order` | observed=['lookup_order', 'issue_refund'] expected=['lookup_order', 'issue_refund'] |
| pass | `final_answer_contains` | expected substring `once` in `Refund completed once.` |
| pass | `max_retries` | retry counts={'lookup_order': 1, 'issue_refund': 1} max_retries=0 |
| pass | `forbidden_tools` | forbidden used=[] |
## Sequence
```mermaid
sequenceDiagram
participant Agent
Agent->>Tool: lookup_order
Tool-->>Agent: lookup_order
Agent->>Tool: issue_refund
Tool-->>Agent: issue_refund
Agent-->>User: final answer
```import json
from tool_call_replay.replay import ReplayCase, ReplayEvent, run_assertions
def test_tool_call_replay_regression():
payload = json.loads('{"events": [{"arguments": {"order_id": "A100"}, "error": null, "output": null, "tool_name": "lookup_order", "type": "tool_call"}, {"arguments": {}, "error": null, "output": {"paid": true, "status": "delivered"}, "tool_name": "lookup_order", "type": "tool_result"}, {"arguments": {"amount": 42.5, "order_id": "A100"}, "error": null, "output": null, "tool_name": "issue_refund", "type": "tool_call"}, {"arguments": {}, "error": null, "output": {"refund_id": "R1", "status": "created"}, "tool_name": "issue_refund", "type": "tool_result"}, {"arguments": {}, "error": null, "output": "Refund completed once.", "tool_name": null, "type": "final_answer"}], "final_answer": "Refund completed once.", "goal": "Refund order A100 without issuing duplicate credits.", "schema_version": "tool-call-replay/v1", "trace_id": "refund-fixed-001"}')
case = ReplayCase(
schema_version=payload["schema_version"],
trace_id=payload["trace_id"],
goal=payload["goal"],
events=[ReplayEvent(event_type=item["type"], tool_name=item.get("tool_name"), arguments=item.get("arguments"), output=item.get("output"), error=item.get("error")) for item in payload["events"]],
final_answer=payload.get("final_answer"),
)
results = run_assertions(case, json.loads('{"final_answer_contains": "once", "forbidden_tools": ["delete_order"], "max_retries": 0, "tool_order": ["lookup_order", "issue_refund"]}'))
assert all(result.ok for result in results), [result.detail for result in results if not result.ok]What you can do
Convert recorded events into one versioned replay format.
Check tool order, forbidden tools, retry limits, and final-answer content.
Emit a self-contained test that fails if the behavior regresses.
See how the product moves from setup through a useful output.
01
Install tool-call-replay 0.1.1 from the package registry.
Package version
02
Run the quick start on the synthetic fixture and compare it with the output above.
Captured output
03
Point the same command at your own files, locally or in CI.
Local report
Release specifications
Review the current source, release, runtime, and support information.
Availability
Review the current version, source, packages, downloads, supported platforms, and install options.
PyPI 0.1.1 is available with a linked source tag v0.1.1.
When the work grows
Continue exploring
Each one lists its source, current version, and install path.