Prompt Caching (Part 2): Five Vendors, Three Clouds, One Bill

Verdict, before the tables.

Move one model to another platform and the multipliers may be identical, the behaviour will not be, and the documentation itself may be wrong.

Claude Sonnet 4.5’s cache multipliers on Bedrock match Anthropic’s first-party API word for word — 1.25× write on the five-minute tier, 2× on the one-hour tier, 0.1× read. The real differences are in three places: the minimum cacheable prefix, which models get a one-hour tier, and whether any diagnostics exist. And failure in all three places is silent.

On that minimum prefix, I originally wrote what the documentation says: 1,024 on the first-party API, 2,048 on Azure, 4,096 on Bedrock, a 4× spread. Then I ran twenty-odd bisecting calls on Bedrock, and the measured threshold is 1,024 — the same as the first-party API. That row of the table is off by 4×. Full data in part four.

So in the tables below, anything I measured is labelled as measured, and anything I did not is labelled as documented. Read the two categories differently, because they do not deserve the same confidence.

If you are choosing between clouds or migrating, this gives you four comparison tables and a per-platform checklist. If you only use one vendor, skip to that section and find out whether the knob you want exists at all.

Everything here comes from vendor documentation current in mid-August 2026 plus my own measurements, cited at the end. Pricing in this area has an alarmingly short half-life — on the day I was compiling these figures, DeepSeek changed its billing model and Azure was changing too. Verify before you copy a conclusion.

Part one covered the mechanism: what is saved is prefill compute, the prefix has to match token for token, and the TTL is a sliding window. This part assumes that.

Current multipliers, all in one place#

Here are the three numbers that matter for five model vendors. Multipliers are relative to the base input rate.

Two things stand out.

First, OpenAI breaks across generations. Before GPT-5.6, cache writes were free and residency could be stretched to 24 hours with one parameter. From 5.6, writes cost 1.25× and the TTL is locked at 30 minutes (prompt_cache_options.ttl accepts exactly one value, 30m). Same vendor, two generations, completely different cost models.

Second, DeepSeek’s read/write ratio is the most extreme of the group. A hit costs about 1/30 of a miss, where everyone else is around 1/10. That ratio is what decides how long a cache is worth keeping alive, which the fourth section works through.

A widely repeated inference that does not hold#

Maxim Khailo’s cross-vendor keepalive study gives a genuinely useful formula:

break-even idle time  T ≈ τ · (w/r − 1)
   τ = ping interval, w = recompute multiplier, r = read multiplier

The formula is right. The step where it is applied to DeepSeek is not. The original argues that DeepSeek’s cold recompute costs only a couple of cents, so no matter how you tune the interval, pinging costs more than the eviction it prevents.

The problem: w and r are both multiples of the same base rate, so absolute cheapness cancels out of the ratio. Recomputing is cheap, and pinging is cheap in exactly the same proportion. Where the break-even sits on the time axis is decided only by w/r; the absolute price decides how large the bill is, not where the crossover falls.

And DeepSeek’s w/r is the largest of the four. By the formula’s own logic, DeepSeek should have the widest profitable range, not “never worth it”.

This is not a gotcha so much as a habit worth keeping: when you see “it is cheap, so it is not worth optimising”, check whether the cheap thing is cheap on both sides of the comparison.

Minimum prefix: keep measured and documented apart#

This section originally read “one model, three platforms, three thresholds”, all quoted from documentation. Then I measured, and it needed correcting.

Measured on Bedrock — two bisecting calls per model, the criterion being either cacheWrite or cacheRead non-zero in usage:

All four models really threshold at 1,024. The only documented figure that disagrees with measurement is the Sonnet 4.5 row. Three control models matched, which says the method is not drifting and Bedrock’s documentation is not broadly unreliable — that one value is wrong. The bisecting procedure and what it cost are in part four.

I did not measure the other platforms, so what follows is documented values. Given the above, verify before you depend on them.

First-party API, by model (documented):

Note that Haiku 4.5 is documented at 4,096, eight times Opus 5’s 512 — the cheapest model carries the highest threshold. Given the Sonnet 4.5 precedent, that number is also worth checking yourself.

Claude on Azure AI Foundry is documented at 2,048. On the Gemini side it climbs by generation:

Gemini 2 series (2.5 Pro / Flash / Flash-Lite)      2,048
Gemini 3 series (3.5 / 3.6 / 3.7 Flash)            4,096
Gemini 3.0 Flash Preview / 3.1 Pro Preview         6,144

Verifying takes two requests#

This is the most practical thing to come out of the measurement: take a prefix sized near the threshold you suspect, send identical content twice, and read the usage. The first call should show a write, the second a read. Both zero means you are under the threshold.

Two requests, a few cents, one minute. Set against the cost of being wrong by 4× and having caching quietly do nothing for a year, that is not an investment worth skipping.

To find out how many tokens your system prompt plus tool definitions actually come to, measure it with the token counter before deciding which range to probe.

Silent failure is the real hazard here#

Whatever the threshold is, the behaviour below it is the same: inference succeeds, the prefix is not cached, no error is returned.

The only detection route is the usage block in the response, and every vendor names the fields differently:

Anthropic   usage.cache_creation_input_tokens
            usage.cache_read_input_tokens

Bedrock     usage.cacheWriteInputTokens
            usage.cacheReadInputTokens
            total input = inputTokens + cacheRead + cacheWrite

OpenAI      usage.input_tokens_details.{cached_tokens, cache_write_tokens}
Azure       usage.prompt_tokens_details.{cached_tokens, cache_write_tokens}
Gemini      usage.total_cached_tokens / cachedContentTokenCount

In all of them, inputTokens / input_tokens means only the part that was neither read nor written — not the total. This is the easiest place to get costs wrong: you think you are looking at the total and you are looking at the remainder. In my measurements a 4,401-token prefix reported inputTokens of 15 on a hit.

And diagnostics exist on exactly one platform#

Anthropic offers Cache Diagnostics (beta, header cache-diagnosis-2026-04-07): the API compares two adjacent requests and tells you which block the prefix diverged at, with a typed reason — model_changed, system_changed, tools_changed, messages_changed and so on.

The limitation is stated in the documentation: Claude API only, not supported on Bedrock or Google Cloud.

Put that next to the previous section and the result is a little ironic: the platform with the least reliable threshold number, and the same silent failure mode as everyone else, is also the one where you cannot get the diagnostic tool.

Residency control has four shapes, and pinging is the worst of them#

This is the section I think is most worth writing down.

The community’s default move for keeping a cache warm is “send a ping on a timer to refresh the TTL”. But laying five vendors’ documentation side by side shows that “keep this cache alive” has four structurally different implementations, and sending pings is the crudest of them:

Anthropic        residency is BOUGHT     5m (1.25×) or 1h (2.0×) → work out which tier
OpenAI ≤5.5      residency is a FREE     prompt_cache_retention: "24h" → no pinging at all
                 switch
Azure (12 models) same                   same parameter, with a published support list
OpenAI 5.6+      residency is FIXED      30m, and writes now cost → pinging turns risky
Gemini explicit  residency is RENTED     storage billed by token-hour, no TTL ceiling
Bedrock          inherits the model,     the 4.6 series has no 1h tier → pinging may be
                 minus some tiers        the only option left

So the right question is not “how often should I ping” but: does this platform give me a residency knob? If it does, turn the knob. Only ping when it does not.

Vendor by vendor:

Anthropic — which tier to buy has a clean crossover. The official multipliers give it to you directly. Let the pause be T minutes, in units of the base input rate:

five-minute tier + a ping every 4 min     0.025 × T
one-hour tier (no pings within 60 min)    an extra 0.75× (2.0 less 1.25)

These are equal at T ≈ 30 minutes. So:

pause < 30 min      five-minute tier, ping about every 4 min
pause 30–60 min     one-hour tier, one write, no pinging
pause > 60 min      one-hour tier, ping about every 50 min

OpenAI ≤5.5 and that batch of Azure models — one free parameter is all you need. Set prompt_cache_retention: "24h" and residency goes from “cleared after 5–10 minutes of inactivity, one hour at most” to up to 24 hours. The documentation is explicit that for models supporting both policies, the two are priced the same. Azure lists twelve supported models, and gpt-5.5 defaults to the extended tier.

On these models the entire question of keepalive intervals does not exist.

Azure has one more lever that almost nobody mentions: on provisioned throughput (PTU-M) deployments, cached input gets up to a 100% discount — free. The cost is that cache breakpoints are unsupported and cache_write_tokens is not reported. If you have already bought PTU, this saving is sitting there.

Vertex explicit caching: where that framework does not fit#

Gemini’s explicit cache is the only one billed by how long you hold it: a CachedContent object accrues storage cost by token-hour, TTL defaults to 60 minutes, the minimum is 1 minute, and there is no maximum — the documentation’s own words are “There isn’t a maximum cache duration” — and you can extend it with patch.

Current storage rates (the pricing page column reads Price Tok/hr, per token per hour):

Gemini 2.5 Pro / 3.1 Pro            $0.0000045 /token/hr  =  $4.50 per 1M token-hours
Gemini 2.5 / 3.5 / 3.6 / 3.7 Flash  $0.000001  /token/hr  =  $1.00 per 1M token-hours

Run a 100k-token prefix through that and the result is counter-intuitive:

Gemini 2.5 Pro
  hold for one hour     100,000 × $0.0000045      = $0.45
  one cold recompute    100,000 / 1M × $1.25      = $0.125

Holding it for an hour costs 3.6× more than simply recomputing it.

Which says Vertex explicit caching is not pause insurance — it is a high-frequency reuse tool. Its economics have nothing to do with pinging:

explicit cache total = storage rate × tokens × hold time + N × read price
no cache total       = N × standard input price

Solving for the break-even hit count N:

Gemini 2.5 Pro     $4.50 / ($1.25 − $0.125)  = 4.0
Gemini 2.5 Flash   $1.00 / ($0.30 − $0.03)   = 3.7

The answer is unusually tidy: at current prices, Vertex explicit caching needs roughly four hits per hour to break even, and Pro and Flash come out nearly the same. Hold it two hours and you need eight.

Note there is no τ in that formula — because nothing is being pinged. The T ≈ τ(w/r − 1) relation quoted earlier assumes holding cost equals ping cost, and here the structure simply differs. That is the boundary of the original framework: none of the four vendors it tested billed by token-hour, so the case never came up.

Gemini implicit caching is a different matter again: there is a published upper bound — the announcement says implicit caches are cleared within 24 hours or less — but no lower bound is committed to at all, only heuristics: put large, general content first, send similar prefixes close together in time. So there is no retention curve you can measure and rely on. That lines up exactly with Khailo’s finding that Gemini’s hit rate behaves like a routing lottery rather than a retention curve — black-box measurement on one side, official silence on the other.

Cross-region inference: three of four are silent on the question that matters#

Cross-region routing dispatches a request to whichever region is optimal, for availability and throughput. A cache is bound to a specific machine or region. Architecturally those two things are in conflict.

OpenAI explains the mechanism most fully, and the explanation is the answer:

• The cache lives in one machine’s GPU-local storage. The documentation states that caching applies when two requests share a prefix and land on the same machine.
• Routing is decided by a hash of roughly the first 256 tokens of the prompt; prompt_cache_key is combined with that hash to raise the chance of landing on the same engine, but the docs call it best-effort and guarantee nothing.
• There is a soft ceiling of about 15 requests per minute per prefix + key combination. Exceed it and nothing errors and nothing is refused — the system spreads the overflow across more machines, and every new machine is a one-time miss.

That last one is a genuine trap for agent fleets: if every instance shares one cache key, your hit rate degrades as you scale, and nothing tells you. At high concurrency you have to shard by key while keeping the key-to-prefix mapping stable.

Bedrock acknowledges that cross-region increases cache writes. The documentation lists “Prompt Caching with Cross-region Inference” as a supported combination while noting that during periods of high demand these optimisations may result in increased cache writes.

What exactly that means is not expanded on in the core documentation. An AWS re:Post article by a Sr TAM adds an explanation: caches are per-Region, and a request routed to one region will not hit a cache built in another — so each region writes its own copy.

But be clear about the source hierarchy: that sentence is on re:Post, not in the core documentation. I had two independent research passes look for it, and neither found any explicit statement in the core docs about whether cache scope is regional or per-machine. So the accurate framing is: AWS acknowledges that cross-region increases cache writes, but has never published the scope of the cache. The re:Post explanation is plausible and comes from an AWS employee, which makes it a strong lead, not a contract.

Nor does AWS recommend switching to a single-region endpoint for hit rate; the advice is to monitor both in-region and cross-region metrics.

Azure and Vertex say nothing at all on the point that matters.

The only clear statement from Azure is that caches are not shared across Azure subscriptions. Whether a Global deployment within one subscription shares caches across regions, and whether you should pick Regional over Global for hit rate — not in the documentation.

Vertex is more contradictory: the docs say context caching supports the global endpoint, but CachedContent is a region-scoped resource (the location is in its path) and the cache is stored in the region the creating request went to. Whether a request routed elsewhere via the global endpoint can hit it — also unwritten.

So what this section can deliver is not a checklist but a judgement: cross-region routing and cache hits are architecturally in tension, and three of four vendors commit to no behaviour whatsoever. You have to measure your own effective hit rate from the usage fields. Handing over a best practice that pretends to certainty would be worse than saying so.

Data residency makes caching more expensive too#

Two surcharges that are easy to miss:

• Anthropic — from Claude 4.6 onward, specifying US-only inference via inference_geo applies a 1.1× multiplier to every token billing category, cache writes and cache reads explicitly included.
• OpenAI — regional processing endpoints add 10% for models released after 2026-03-05.

Which means that once data residency is on, every keepalive ping is 10% dearer as well. Do not leave it out of a cost estimate made under compliance constraints.

Per-platform checklist#

Everything above, compressed into actions.

Anthropic first-party API. Pick the tier by pause length: under 30 minutes, five-minute tier with a ping roughly every 4 minutes; 30–60 minutes, buy the one-hour tier and do not ping; over an hour, one-hour tier with a ping roughly every 50 minutes. Use max_tokens: 0 for the ping — the official warm-up shape, output not billed, replacing the old max_tokens: 1 workaround. Turn on Cache Diagnostics when investigating. Remember the TTL is counted from the start of the request and streaming time counts, so your real interval has to be shorter than the theoretical one.

AWS Bedrock. Measure the minimum prefix yourself rather than copying the table — I measured 1,024 for Sonnet 4.5 where the docs say 4,096. Then check whether the model you want has a one-hour tier at all: on Bedrock, Opus 4.6 and Sonnet 4.6 only have five minutes. The minimum token count is evaluated across tools → system → messages combined, not per block. Treat cross-region inference as a controlled variable when running cache experiments. Caching is not supported with the batch inference API. GPT-5.6 on Bedrock does not go through the Converse API — sending a cachePoint returns AccessDeniedException; use invoke-model with prompt_cache_breakpoint.

Azure OpenAI / AI Foundry. Check which generation your model belongs to. GPT-5.5 and earlier: one prompt_cache_retention: "24h" is all you need, and both tiers cost the same. GPT-5.6+: the TTL is locked at 30 minutes and writes now cost, so put an explicit breakpoint after your stable content and use explicit mode to keep the volatile tail out of the cache. If you already run PTU-M, cached input gets up to a 100% discount. Claude on Foundry uses Anthropic’s native cache_control, with a 2,048 minimum prefix, and cache reads do not count toward ITPM.

Google Vertex AI. Decide first whether your pattern is high-frequency reuse or long pauses. High-frequency reuse — more than about four hits an hour — justifies explicit caching, with the TTL set to whatever you need since there is no ceiling. Long pauses or infrequent access: do not use explicit caching, the storage fee will eat the benefit. Implicit caching needs no configuration and guarantees nothing; putting large, general content first is the whole of what you can do. Note the Gemini 3 series threshold has risen to 4,096, and 6,144 on preview models.

OpenAI first-party API. Set prompt_cache_key — on GPT-5.6 it is required for reliable matching — and shard on roughly 15 requests per minute per key. On models ≤5.5, just turn on 24-hour retention. On 5.6+, use explicit mode to keep the volatile tail outside the breakpoint so you are not paying for writes repeatedly. Watch for cache_write_tokens staying high while cached_tokens stays low; that is the signature of a breakpoint in the wrong place.

DeepSeek. The read/write ratio of about 1/30 is the most favourable among the major vendors, so caching is worth using. But it publishes no retention commitment, so you have to measure it. Also, from 2026-08-16 it moved to time-of-day pricing, with off-peak at half the peak rate — and the peak windows (01:00–04:00 and 06:00–10:00 UTC) land squarely on 09:00–12:00 and 14:00–18:00 in China, covering the domestic working day precisely. Moving batch jobs, CI and regression suites to the off-peak window halves that spend outright.

In short#

Every number above uses a 100k-token prefix as its example, and your prefix length, hit frequency and pause distribution are all different from mine. Putting your own figures through the LLM cost calculator beats copying anyone’s conclusion — particularly for that Vertex “four hits an hour” threshold, which moves with the model and the prefix length.

Move a model to another platform and the price may be identical, the behaviour will not be — and the documentation may be wrong. Claude Sonnet 4.5’s cache multipliers are the same on Bedrock and the first-party API, and I measured the minimum prefix at 1,024 on both, against 4,096 in the AWS table. The real differences are which models get a one-hour tier, whether diagnostics exist, and the fact that failure is silent everywhere.

Look for the knob before you reach for pings. Residency control comes in four shapes — buy a tier, flip a free switch, locked and unadjustable, or rented by token-hour. Pinging is what you do when there is no knob, not the default.

Billing by token-hour changes the whole model. Vertex explicit caching needs roughly four hits per hour to break even; it measures reuse frequency, not pause length.

Cross-region routing and cache hits are inherently in tension, and most vendors commit to nothing. Measure your effective hit rate rather than trusting an inference.

Part four is the empirical companion to this one: I bisected the threshold for four models on Bedrock, and one disagreed with its documentation. Part three covers the engineering failures that actually destroy a cache — non-deterministic serialisation order, tool ordering, proxies silently stripping cache markers — and how to build a self-check script out of the usage fields.