Home / Case Studies / Human-Gated Agent Orchestration

Human-Gated Agent for Refund Automation

Plan, act, observe, with schema validation at the boundary and a hard approval gate in front of every irreversible action.

Agents Tool Calling Human-in-the-Loop Pydantic Observability tau-bench

TL;DR

The Decision That Triggered This Build

A returns desk is the canonical case for agents, multi-step, tool-dependent, high-volume, and the canonical case against them, because the final step moves real money out of the business. Automating the lookup is obviously good. Automating the payout is obviously not.

The product question: which steps can run unsupervised, and what does the boundary cost in throughput? An agent that asks permission for everything is a slow form; one that asks for nothing is an unbounded liability.

What an Agent Actually Is

Tools

Plain functions the model may request, each with a typed schema.

A loop

Ask the model, run what it asked for, feed the result back, repeat.

State

An ordered history, so a run can be inspected, resumed, replayed.

A stopping rule

Task complete, or step budget exhausted.

Everything else, the framework, the plan/act/reflect vocabulary, is engineering to make those four reliable.

Technical Approach

Agent control-flow diagram. The model is asked for the next action; if it requests no tool the run finishes. Each tool call passes schema validation first, and invalid calls return an error observation the agent can correct from. Valid calls are checked for high-risk status, ordinary tools execute directly, while high-risk tools stop at a highlighted human approval gate that either permits execution or records a refusal. A footer notes that every node writes an ordered trace with a run id and step number.

Schema validation and a human approval gate stand between the agent and any irreversible action. Measured: 100% of high-risk attempts gated.

Typed tools

Pydantic models already emit JSON Schema, so the API tool definition and the runtime validator are the same declaration. No second definition to drift.

Validation failures return as data

A malformed call comes back as an observation the model reads and corrects on the next turn. An exception ends the run; an observation lets it recover.

Ordered traces

Shared run_id, monotonic step. Timestamps are not reliable ordering when steps land in the same millisecond, and replay is the only real way to debug an agent.

Design choice: the gate sits after validation, before execution

Approving an unvalidated call means the human is approving something that may not even be well-formed. Validate first, surface the concrete action, then execute. A refusal is recorded as a normal outcome the agent can see and respond to, not an exception. The provider's automatic function-calling is deliberately disabled so the loop executes tools itself and the gate stays inside it, verified by driving a refund on a policy-ineligible order and confirming the gate blocked it and no money moved.

Evaluation

Task-success rate
1.00
4 / 4 tasks
Tool-call accuracy
1.00
mean across tasks
High-risk actions gated
1.00
1 attempt, 1 gate fired
Cost per task
$0.00063
mean latency 23.8 s

Per-task cost is measured from token usage, and the full tau-bench sweep is projected from it before being run, roughly 115 tasks × the measured $0.00063 ≈ $0.07 for the full suite. That projection is why development happens against a small subset and the full suite runs once. To be explicit: this is a 4-task fixture, not tau-bench, the number is a proof the machinery works, not a benchmark result.

Main weakness: the gate relocates judgement, it does not remove it

An approver who says yes to everything lets a bad refund through; one who says no to everything blocks the legitimate one too. There is no setting that is simply correct. The same shape recurs in the semantic cache's threshold, the text-to-SQL confidence cutoff, and the retrieval judge's bias, naming that recurrence across four projects is the through-line of the whole set.

Two things visible in the per-task data are worth stating. Task success and tool-call accuracy can diverge even when both read 1.0 here, a run can reach the correct outcome while calling a tool wrong and silently recovering. And latency varied 3.3 s to 68 s across the four tasks (the thinking model plus free-tier rate-limit backoff), so a single mean hides a wide spread.

Recommendation

Gate on the tool, not the step

Read-only tools run freely; money-moving and destructive tools always stop. One approval per refund removes the entire category of unsupervised financial error, measured here as 100% of high-risk attempts gated.

Next Iteration Priorities