Skip to content
Sentynel

#technical4 min readBy Sentynel

Cost per resolved case: the one unit metric that tells you if an agent pays off

Token bills and hours saved do not show whether an agent works. Cost per resolved case does. Here is how to define it, measure it and act on it.

Share

Most teams track what an agent costs (tokens, licences, build hours) or what it did (messages sent, tasks started). Neither tells you whether the operation got better. The metric that does is cost per resolved case: everything the operation spent on a unit of work, divided by the units that actually reached a finished state.

Why the usual numbers mislead

  • Cost per call or per token falls every time a model gets cheaper, and the operation can still be worse. A cheap agent that sends half the cases back to a person is expensive.
  • Hours saved is an estimate of work that did not happen. Nobody can audit it.
  • Volume handled counts activity. A case that is opened, bounced and reopened is counted three times.

All three measure the agent. The business pays for the operation, so the unit has to be the case, and the denominator has to be the cases that ended well.

Define the case and the finish line first

Before any arithmetic, write two sentences:

  1. What is one case? An enquiry, an order, a report, a ticket. One record with an identifier and a start.
  2. What counts as resolved? A named end state in your pipeline that someone owns, with a rule for what is excluded (duplicates, spam, cases reopened within a set window).

If you cannot name the end state, you do not have a metric yet. You have a mood.

Count every cost that touches the case

The numerator is the full cost of the operation for that period, not the agent's invoice:

  • model and platform usage;
  • the people who review, correct or take over cases, at their real time;
  • the cost of the exceptions that left the automated path;
  • maintenance: prompt, tool and data upkeep, spread across the period.

Then divide by resolved cases in the same period. Keep the cohort the same on both sides. If the numerator covers a month, the denominator covers the cases created in that month, resolved by the end of the window you declared.

How the metric is built
  1. 01Define the caseOne record, one identifier
  2. 02Name the end stateResolved, with exclusions
  3. 03Sum every costModel, people, exceptions, upkeep
  4. 04Divide by resolvedSame cohort, same window

Review monthly and compare against the previous window, not against a vendor claim.

Read the metric in three parts

A single number hides the cause. Split it:

Part Question What moves it
Resolution rate Of the cases created, how many reached the end state? Clearer procedures, better exception routing
Touch cost How much human time does a resolved case consume? Review rules, handoff quality
Run cost What does the automated part cost per case? Model choice, step count, retries

Most improvements come from the first two. Run cost is the part vendors talk about, and it is usually the smallest lever once the flow is working.

Rules that keep the number honest

  • Fix the cohort before you look. Decide the window and the exclusions in advance, so nobody can redraw them after seeing the result.
  • Report the first window as a first window. With no earlier baseline, the figure is a starting point, not a trend.
  • Show the exceptions. The cases that left the automated path stay in the numerator and in the report.
  • Keep the case record. Every number should trace to records someone can open.

A Sentynel example

In the Rodo operation, the reported figure is that 82% of live-created enquiries reached Resolved. That is a resolution rate: the first part of the metric, measured on a defined cohort. The source is Rodo's operational report of 24 September 2026; the resolution cohort covers 31 August to 24 September, and the figures describe pipeline records, not all conversations. It is a first measured operating window, which is exactly how it should be read: a baseline to compare the next window against.

Rodo · first measured window
of live-created enquiries resolved

Source: Rodo operational report, 24 September 2026. The resolution cohort covers 31 August–24 September. Figures describe pipeline records, not all conversations.

A test for this week

Pick one process an agent touches. Write the case definition and the end state in two sentences, then list the four cost lines above with a rough owner for each. If you cannot fill one of them, that gap is the first thing to fix, before any model upgrade.

Which of your agents could you put a cost per resolved case on today?

Share
← All sygnals