Measure the agent, then change it.
One support agent looks up orders, issues refunds, and changes addresses on a fresh mock database. Three projects measure what happens when it loops, when a refund is retried, and when the same work is split across workers. Open the playground to watch one task, then read the measured before-and-after on Results.
Run a taskHow a visit works
- Playground
- Agent loop
- Tools
- Fresh database
Guards and the live trace sit on the loop. The full eval set runs on a laptop and is saved as JSON. This site replays one task at a time and displays those saved numbers.
Guardrails
A step limit, a cost budget, loop detection, and a human ticket when a run gets stuck.
Stuck rate 0.0% → 0.0%
Trajectory and refunds
Score the tool path separately from the final database, and make a timed-out refund safe to retry.
Measured comparison not published yet.
Single vs multi-agent
The same harder tasks, solved by one agent and by a lead that delegates to workers.
Measured comparison not published yet.