Prompt caching cost savings, explained

Prompt caching (also called context caching) lets a provider reuse the processing it already did on a chunk of input you've sent before, and charge you a fraction of the standard input rate for that reused portion instead of the full price. It exists because a large share of real-world LLM traffic re-sends the same static content on every call — a system prompt, a set of tool definitions, a long reference document — with only a small user query changing between requests. Without caching, you pay full price for that static block every single time.

Which providers support it

Caching is now common across the major labs, but the mechanics differ: some cache automatically based on a matching prefix, others require you to explicitly mark a cache breakpoint, and the discount depth and cache lifetime both vary by provider and sometimes by model. Check a specific model's page for its cached input rate — models.json in this dataset carries a dedicated field for it, and a model with no listed cached rate simply doesn't support caching yet.

What actually makes a request cache-eligible

• The cached portion has to be a stable prefix — identical bytes, identical order — not just similar content. A system prompt with a timestamp or session ID inlined at the top defeats caching for the whole block.
• Put static content first and variable content last in your prompt structure, so the matching prefix is as large as possible.
• There's usually a minimum token count for a block to be cache-eligible at all — very short system prompts may not qualify regardless of provider.
• Cache entries expire after a period of inactivity; a workload with long gaps between requests may not benefit as much as a tight request loop.

Cache lifetime: it doesn't stay warm forever

A cached block expires after a period without a matching request — the exact window is set by the provider and isn't unlimited. For a request pattern with steady, frequent traffic, the cache stays warm continuously and the discount applies on nearly every call. For a bursty or low-frequency workload, the cache can go cold between uses, and you end up paying the write cost again on the next request without ever collecting the discount. Measure your actual request cadence before assuming caching will pay off — it's a much better fit for tight loops than for occasional, spaced-out calls.

Where the savings actually show up

Caching only discounts the input side — cached tokens are still input tokens, priced lower than standard input but never free, and output tokens are unaffected entirely. That means caching helps most on workloads with a large, static context and a short generated response: RAG systems re-sending retrieved context, coding agents re-sending a file tree or long file contents, multi-turn chat re-sending conversation history. It does very little for a workload that's mostly output — long-form generation from a short prompt won't see much benefit.

Run your prompt shape through the calculator with and without caching enabled to see the actual dollar difference for your workload — the savings depend entirely on how much of your input is static.

The first-write cost

Several providers charge a small premium the first time a block is written to cache, before any discount applies on subsequent hits. For a request pattern that reuses the same context many times, that write cost is trivial against the savings; for a pattern where every prompt is genuinely unique, caching adds cost rather than saving it. If your traffic doesn't repeat prefixes, skip it — caching is a targeted optimization, not a default to flip on everywhere.

The practical test is straightforward: log how often your actual production prompts share an identical prefix with a prior request within the provider's cache window. If that number is high, structure your prompt so the static portion comes first and enable caching. If it's low, the discount won't materialize often enough to matter, and your time is better spent on model selection or batch eligibility instead.