How do I calculate cost per AI outcome?

Scope

Use this method for a workflow with an observable outcome, such as a case resolved, document accepted, incident triaged, or approved action completed. Do not use “model response,” “token,” or “session” as the business outcome unless that is truly what the user buys or values.

Why it happens

Provider invoices and token counters are easy to collect, while outcome data lives in product systems. Teams therefore divide spend by requests and call the result unit economics. That hides repeated calls, failed tool execution, escalations, review labor, rejected output, and reversals. It also rewards designs that emit fewer expensive requests even when they deliver fewer acceptable outcomes.

Diagnosis

1. Write the outcome in past tense: “case resolved and accepted,” not “answer generated.”
2. Define the attribution window and the event that closes it.
3. Name accepted, rejected, abandoned, reversed, and unknown states.
4. Trace every request and effect to a stable outcome ID.
5. Reconcile model and tool costs with invoices or governed usage exports.
6. Choose and document shared platform and labor allocation rules.
7. Count records that lack outcome identity or cost identity; report them as coverage gaps.

Solution

Maintain a request-to-outcome ledger at the finest governed level needed for attribution. Aggregate only after reconciliation. For a period and scope:

cost per accepted outcome = fully loaded cost of all attributed attempts / accepted outcomes

Publish companion measures: acceptance rate, reversal rate, unknown-attribution rate, human-review share, retry share, and serious-failure count. Segment by task family or user cohort before comparing routing or model policies. A global average can hide the workflow that consumes the most retries or produces the most expensive reversals.

Artifact

Download the cost-per-outcome ledger. Its rows are synthetic and include metadata, accepted, rejected, and reversed examples. Replace the example outcome type, cost sources, quality gate, and allocation notes with approved local definitions.

Minimum fields are outcome ID and type, status, request count, model cost, tool cost, retry cost, human-review cost, allocated platform cost, total cost, quality gate, period, and attribution note.

Common mistakes

• Dividing total tokens by successful requests while dropping failed and retried work.
• Comparing two models before holding task mix and acceptance criteria constant.
• Treating an outcome proxy as revenue or causal business value.
• Allocating all shared cost by request count when workload intensity differs materially.
• Rewriting historical outcomes after a reversal without preserving the original record and timing.