Cost per resolved case: the one unit metric that tells you if an agent pays off
Token bills and hours saved do not show whether an agent works. Cost per resolved case does. Here is how to define it, measure it and act on it.
Most teams track what an agent costs (tokens, licences, build hours) or what it did (messages sent, tasks started). Neither tells you whether the operation got better. The metric that does is cost per resolved case: everything the operation spent on a unit of work, divided by the units that actually reached a finished state.
Why the usual numbers mislead
- Cost per call or per token falls every time a model gets cheaper, and the operation can still be worse. A cheap agent that sends half the cases back to a person is expensive.
- Hours saved is an estimate of work that did not happen. Nobody can audit it.
- Volume handled counts activity. A case that is opened, bounced and reopened is counted three times.
All three measure the agent. The business pays for the operation, so the unit has to be the case, and the denominator has to be the cases that ended well.
Define the case and the finish line first
Before any arithmetic, write two sentences:
- What is one case? An enquiry, an order, a report, a ticket. One record with an identifier and a start.
- What counts as resolved? A named end state in your pipeline that someone owns, with a rule for what is excluded (duplicates, spam, cases reopened within a set window).
If you cannot name the end state, you do not have a metric yet. You have a mood.
Count every cost that touches the case
The numerator is the full cost of the operation for that period, not the agent's invoice:
- model and platform usage;
- the people who review, correct or take over cases, at their real time;
- the cost of the exceptions that left the automated path;
- maintenance: prompt, tool and data upkeep, spread across the period.
Then divide by resolved cases in the same period. Keep the cohort the same on both sides. If the numerator covers a month, the denominator covers the cases created in that month, resolved by the end of the window you declared.
- 01Define the caseOne record, one identifier
- 02Name the end stateResolved, with exclusions
- 03Sum every costModel, people, exceptions, upkeep
- 04Divide by resolvedSame cohort, same window
Review monthly and compare against the previous window, not against a vendor claim.
Read the metric in three parts
A single number hides the cause. Split it:
| Part | Question | What moves it |
|---|---|---|
| Resolution rate | Of the cases created, how many reached the end state? | Clearer procedures, better exception routing |
| Touch cost | How much human time does a resolved case consume? | Review rules, handoff quality |
| Run cost | What does the automated part cost per case? | Model choice, step count, retries |
Most improvements come from the first two. Run cost is the part vendors talk about, and it is usually the smallest lever once the flow is working.
Rules that keep the number honest
- Fix the cohort before you look. Decide the window and the exclusions in advance, so nobody can redraw them after seeing the result.
- Report the first window as a first window. With no earlier baseline, the figure is a starting point, not a trend.
- Show the exceptions. The cases that left the automated path stay in the numerator and in the report.
- Keep the case record. Every number should trace to records someone can open.
A Sentynel example
In the Rodo operation, the reported figure is that 82% of live-created enquiries reached Resolved. That is a resolution rate: the first part of the metric, measured on a defined cohort. The source is Rodo's operational report of 24 September 2026; the resolution cohort covers 31 August to 24 September, and the figures describe pipeline records, not all conversations. It is a first measured operating window, which is exactly how it should be read: a baseline to compare the next window against.
Source: Rodo operational report, 24 September 2026. The resolution cohort covers 31 August–24 September. Figures describe pipeline records, not all conversations.
A test for this week
Pick one process an agent touches. Write the case definition and the end state in two sentences, then list the four cost lines above with a rough owner for each. If you cannot fill one of them, that gap is the first thing to fix, before any model upgrade.
Which of your agents could you put a cost per resolved case on today?