Skip to content
Sentynel

#technical3 min readBy Sentynel

The ten-step agent: why short, controllable runs beat autonomy in production

A study of agents in production found 68% run at most 10 steps before a person steps in. Here is how to design operations around short, bounded agent runs.

Share

A study of agents running in real organizations found that 68% of production agents execute at most 10 steps before a person intervenes, and that reliability is the top challenge teams report. The lesson is not that agents are weak. It is that the ones that work are designed as short, bounded runs inside a larger operation. This article shows how to do that on purpose.

What the study actually says

"Measuring Agents in Production" (Pan et al., UC Berkeley, Stanford and others; accepted to ICML 2026) interviewed 20 case-study teams and surveyed 86 practitioners across 26 domains. Among its findings: 70% of agents rely on prompting off-the-shelf models rather than tuning them, and 74% depend primarily on human evaluation. The authors summarize the pattern as agents built with "simple, controllable approaches."

Agents in production (survey of 86 practitioners, 20 case studies)
run at most 10 steps before human intervention
use prompting on off-the-shelf models
rely mainly on human evaluation

Source: Pan et al., Measuring Agents in Production, arXiv:2512.04123

The survey is a snapshot of teams that chose to answer it, so read it as a pattern, not a law. But the pattern matches what operations people see: a short run is one you can check.

Why short runs win

  • Errors compound. Each step can be slightly wrong; ten steps leave less room to drift than a hundred.
  • Review is cheaper. A person can read ten steps of output in a minute. A long trace gets skimmed or ignored.
  • Failure is local. When a short run fails, you know where, and only that unit of work is redone.
  • Ownership is clear. A bounded run has an input, an output and a person who accepts it.

Step 1: cut the job at its natural seams

Take one workflow and list its decisions. Every place where the work changes owner, system or risk is a seam. Cut there. Quoting a customer, checking stock and confirming an order are three runs, not one agent called "sales".

Step 2: give each run a contract

Write, for each run, in a few lines:

  1. Input: exactly what it receives, and where it comes from.
  2. Output: a checkable result, saved to a system of record, not left in a chat.
  3. Step budget: a maximum number of steps. Past it, the run stops and escalates instead of improvising.
  4. Stop conditions: missing data, low confidence, a forbidden action.

Step 3: put the handoff where the risk is

A chain of short runs
  1. 01Intake runClassify and record
  2. 02Work runDraft within a step budget
  3. 03CheckRule or person verifies
  4. 04Action runWrite to the system

A run that hits its step budget goes to its owner, not to another retry.

Let reads and drafts run freely. Put the check before anything irreversible or customer-facing: sending, paying, changing a price, deleting.

Step 4: measure the handoffs, then lengthen the leash

Log, per run: steps used, whether it hit the budget, whether the person changed the output. Those three numbers tell you where to extend autonomy. Raise a step budget only when the data shows the checkpoint rarely changes anything. That is how a ten-step agent earns twenty, and it is also how you avoid discovering the limits in front of a customer.

What to do this week

Pick one workflow where an agent already runs. Count its steps in a typical run and in the worst one. If the worst case is far above ten, cut it at the seams above, write the contract for the first run, and have its owner review the output for a week. The operation will tell you where the next cut goes.

Sources

  1. https://arxiv.org/abs/2512.04123
Share
← All sygnals