Your idempotency key cannot prove what you were charged for

A paid API call returns 200. You have a result. What you do not have is proof of how many times you were charged for it.

That gap is where most cost control quietly fails. Teams instrument the easy number — tokens billed, requests per minute, monthly spend — and never instrument the hard one: the correspondence between responses you accepted and charges you incurred. When the two diverge, nothing alerts. The bill just comes in higher and everyone assumes traffic grew.

This is what an idempotency key does and does not do, and what you have to build alongside it.

An idempotency key is a forward-looking promise

When you send Idempotency-Key: K, you are asking the provider: if you have already seen K, return the earlier result instead of doing the work again. That prevents a duplicate charge for a retry that happens inside the provider's memory.

It proves nothing about the past. Three consequences follow, and each one is a production incident waiting to be scheduled:

1. The protection expires. Providers retain idempotency records for a bounded window — 15 minutes, an hour, a day, whatever the contract says. Outside that window, the same key is a new request. If your retry policy can back off longer than the retention window, then your retries silently become purchases, and the failure is invisible because you get a result either way.

The fix is not cleverness, it is a timestamp. Persist the first send time for each key, and refuse to replay a key older than the documented window. If a job is still unresolved when the window closes, do not retry it — escalate it.

2. The replay header is not a charge detector. Many providers return Idempotent-Replayed: true when serving from the idempotency record. It is tempting to build the obvious assertion: if I sent a retry and the response lacks that header, I was charged twice.

That assertion is wrong, and it fails in the expensive direction — a false positive that trains you to ignore it. A success response without the replay header is exactly what a genuine first execution looks like, when the original request never reached the API at all. Same signature, opposite meaning.

The header tells you "this response came from memory." It cannot tell you "the original work happened." Those are different facts, and only the second one costs money.

3. "Billing is separate from caching" is the normal case. A provider may keep the billing record and the cached response in different stores with different lifetimes. So a request can be charged, its response lost, and its replay refused — typically surfacing as something like 409 replay_unavailable carrying the original request id. That status is not an error to back off from. It is the provider telling you you were charged and the answer is gone. Treat it as a billing event, not a transport event.

Prove it by reconciliation, not by inspection

If no single response field can tell you whether you were charged, stop looking for one. Compare two systems that each know half the truth:

• your ledger — every charge the provider actually applied
• your accepted responses — every result you treated as work performed

The invariant is one-to-one: one charge per accepted response. Join them on the request id and the key. If the counts differ, you have found either a double charge or a lost result, and both are worth waking someone for.

This is a weaker-sounding claim than "assert on the header," and it is the only one that is true. It also has the useful property of failing when a provider changes its retention window, its header behaviour, or its billing semantics — which is exactly when you need to know.

Three practical requirements:

Make the reconciler idempotent. The reconciliation job is itself a job that can retry. If it double-counts its own runs, it will manufacture the discrepancy it exists to detect. Give it a run id and make its writes conditional on that id.

Emit a receipt per attempt, not per run. Record the decision, the amount, the request id, the key, and the policy version that produced the decision. When the rules change, you need to know which receipts were produced under the old rules — otherwise a policy fix makes historical reconciliation unexplainable.

Alert on the rate, not the count. A single mismatch is usually a race that resolves. A rising mismatch rate means the retry policy has drifted into charging territory. Watch mismatches per thousand accepted responses.

What this looks like as an SLO

The number worth putting on a dashboard is not "spend." It is:

charges per accepted response — target 1.00, alert outside [0.98, 1.02]

And beside it:

unresolved attempts older than the retention window — target 0, alert above 0

The second one is the cheapest early warning you will ever build. It catches the case where retries are about to stop being free, before the invoice tells you.

The general lesson

Every "did this happen twice?" question in a paid system has the same shape. An idempotency key reduces the probability of a duplicate. A receipt is the evidence that it did or did not occur. Those are not substitutes, and the substitution is invisible until the bill arrives.

Cost control is not a pricing problem. It is a bookkeeping problem with a pricing surface.

If you are building against multiple model providers and want the ledger-style accounting described here — per-request cost attribution with a recorded decision for every call — that is what we run at aiapi-pro.com: one OpenAI-compatible endpoint over 109 models, with the numbers exposed rather than summarised.

Related: if you are still deciding what a token should cost you, Four cost gates that decide your token bill is the pricing half. This post is the bookkeeping half.