Prompt caching, and when it actually pays

Prompt caching is the largest cost lever most applications never pull. It is also the one most often described as free — which is true on one provider and false on the other, and the difference is worth understanding before you turn it on.

The mechanism is the same everywhere. If the beginning of your request is identical to one the provider processed recently, it can reuse the work instead of redoing it, and charges a reduced rate for that reused portion. What differs — and it changes the entire analysis — is whether you pay a premium to create the cache entry in the first place.

Two providers, two pricing models#

OpenAI applies caching automatically to prompts above roughly 1,024 tokens. There is nothing to configure. A cache miss costs exactly what an uncached request costs, and a cache hit costs about half the normal input rate.

Anthropic requires you to mark what to cache, with an explicit cache_control breakpoint. Writing an entry costs about 1.25x the normal input rate, and reading one costs about 0.1x — a tenth.

Neither set of multipliers is modelled in this tool's comparison table, and both change. Check them yourself before committing to a number: OpenAI's pricing page and Anthropic's. But the shapes have been stable, and the shape is what decides whether caching can hurt you.

OpenAI: there is no break-even#

If a miss costs 1.0 and a hit costs 0.5, then at hit rate h your cost per unit of prefix is (1−h) × 1.0 + h × 0.5, which is below 1.0 for any h greater than zero.

That is the whole calculation. There is no threshold, no minimum traffic, no arrival-rate condition. One hit in a thousand requests saves the price of half a prefix; ninety hits in a hundred saves 45% of your input bill. A prompt that never hits costs exactly what it would have cost anyway. The downside is zero.

On GPT-4o at $2.50 per million input tokens, a 20,000-token stable prefix costs $0.05 per request uncached. Across 100,000 requests a month that is $5,000. At a 90% hit rate you pay $2,750. At a dismal 10% hit rate you still pay $4,750 — less than the $5,000 you were paying before.

So on OpenAI the only question is whether your prompt is long enough to qualify and structured so the prefix actually repeats. There is no version of this where enabling it is the wrong call.

Anthropic: the write premium creates a threshold#

Anthropic's numbers are more aggressive in both directions — the read discount is five times deeper, and there is a penalty for missing.

Suppose a stable prefix of P tokens is sent N times. Uncached you pay N × P. Cached you pay a write on the first request and a read on the rest: 1.25P + 0.1P(N−1). Those cross at N = 2 — two requests cost 1.35 units against 2.0, and ten cost 2.15 against 10, a 78% saving.

That framing assumes every request after the first finds a warm entry. The honest version is expressed as a hit rate. At hit rate h, the cost per unit of prefix is (1−h) × 1.25 + h × 0.1, which beats 1.0 when:

h > 0.25 / 1.15 ≈ 22%

Above roughly a one-in-five hit rate, Anthropic caching saves a great deal — more than OpenAI's, because the read is a tenth rather than a half. Below it, the write premium on all those misses outweighs what the reads save. At a 0% hit rate, every request is a fresh write and you are paying 25% more than if you had never enabled it.

This threshold is a property of Anthropic's pricing model, not of caching in general. It does not exist on OpenAI.

The same endpoint, both directions#

Take a retrieval-augmented endpoint with a 20,000-token stable prefix — system prompt, tool definitions, a policy document — on Claude Sonnet 5 at $3 per million input tokens. That prefix costs $0.06 per request uncached.

High traffic. 100,000 requests a month, arriving steadily, 90% hit rate. Uncached: $6,000. Cached: (0.10 × 1.25) + (0.90 × 0.1) = 0.215, so $1,290 — about $4,700 saved by adding a cache breakpoint.

Low traffic. The same endpoint on an internal tool serving 5,000 requests a month, arriving unevenly against a five-minute entry lifetime, so most find nothing warm. At a 10% hit rate: (0.90 × 1.25) + (0.10 × 0.1) = 1.135. Uncached: $300. Cached: $341. You have paid $41 for the privilege.

Nothing about the second case looks wrong in code. It is the same configuration that saved $4,700 in the first one; only the arrival rate changed. Run it on OpenAI and that case saves a little rather than losing a little — which is the practical difference between the two pricing models.

The four things that defeat it#

These apply on both providers. The consequence differs: on OpenAI a defeated cache costs you nothing but the saving you did not get, while on Anthropic it costs you the write premium as well.

A timestamp in the prefix. The match is on an exact prefix. Interpolating Current date: 2026-08-05T14:22:09Z into your system prompt means every request is a miss, forever. So does a request ID, a random greeting, or a user's name near the top. This is by far the most common cause, and the fix is to move the volatile parts after the stable ones.

Ordering. Only the prefix caches — everything up to the first token that differs. A request assembled as [system][user question][retrieved documents] caches the system prompt and stops. Assembled as [system][retrieved documents][user question], it caches both large blocks. Same tokens, same answer, very different bill.

The lifetime. Entries expire after a few minutes of disuse. Steady traffic keeps one warm indefinitely; a workload with gaps keeps re-creating it. On OpenAI that just means fewer hits. On Anthropic it means repeatedly paying to write — and while a longer lifetime is available, it carries a higher write premium, which raises the break-even hit rate rather than lowering it.

Minimum length. Both providers set a floor, around a thousand tokens, below which nothing is cached. A 400-token system prompt is not a caching opportunity no matter how often you send it.

Where it composes with everything else#

Caching pairs unusually well with the two situations that generate the most tokens.

Long conversations are close to the ideal case: the history is a stable prefix that only grows at the end, so every turn after the first is mostly a cache read. It does not stop the total from growing with the square of the length — it reduces the price of the growing part, so it complements summarising or windowing the history rather than replacing them.

Large fixed context — a policy document, a schema, a set of few-shot examples — is the other. If it is identical on every call, it belongs at the front of the prompt.

What caching does not do is make a bloated prompt acceptable. Half price, or even a tenth, is still charged on every request forever. Deleting a redundant 5,000-token block saves 100% of it. Do the deletion first, then cache what survives.

Before you enable it#

On OpenAI, the checklist is short: make sure your prompt clears the length floor and that the stable material comes first. Then stop thinking about it, because there is no scenario in which it costs you.

On Anthropic, three numbers decide it. Measure your prefix length — paste the fixed portion of a real request into the calculator, and if it is under about a thousand tokens, stop here. Compare your request arrival rate against the entry lifetime: if a typical gap between requests sharing a prompt exceeds a few minutes, your hit rate will be poor and the write premium will eat the gain. And diff two real outbound requests token by token to find where they first differ. That position, not your intention, is what gets cached.