This Q&A guide is for developers and product teams investigating OpenAI, Claude or Gemini API costs. It covers provider prompt caching, not ChatGPT or Claude subscription allowances. Numerical examples use hypothetical rates, not current provider quotes or measured customer savings.

What should I check first when caching does not save money?

Separate a cache-hit problem from a cost-accounting problem. “Caching enabled” describes a configuration; it does not establish that requests reused an entry or that the overall workload became cheaper.

Prompt caching symptoms and the next check
SymptomPossible causeNext check
No cached inputIneligible requests, first use or missing configurationModel, endpoint, minimum length, usage fields and a controlled repeat
Repeated writes, few readsChanging prefix or expiry between requestsStable content, tool configuration and request intervals
Many hits, little savingsSmall cached share or high output and tool costsCached tokens and dollars as shares of the total
Input cheaper, invoice higherMore traffic, longer answers or extra stepsCost per comparable task and workload volume
Caching costs moreLow reuse, expensive writes, storage or double countingComplete cost calculation for the cache lifetime

Keep the provider, model, endpoint and billing period fixed when comparing. If the model or workflow changed too, use the task-cost investigation guide to separate usage changes from rate changes.

Why are cached tokens zero even when I send the same question?

The visible question may be the same while the actual input prefix is different or ineligible. Check model support, minimum cacheable length, configuration, cache scope and the time between requests.

OpenAI documents matching-prefix behavior and model-specific requirements in its prompt-caching guide. Anthropic documents cache_control, breakpoints and cache lifetime in its Claude caching guide. Do not transfer one provider’s settings or usage-field interpretation to another API.

What can change the prefix without changing my question?

A timestamp, request ID, reordered tool definitions, dynamic instructions or changing retrieved documents can alter input before the reusable content. Inspect the request your application actually sends, including framework-added context, using an appropriate secure debugging process.

Less reusable layout:
[current timestamp and request-specific data]
[stable instructions and reference document]
[user question]

Candidate layout to test:
[stable instructions and reference document]
[request-specific data and user question]

This is a layout illustration, not a provider request schema or a guaranteed hit. Preserve instruction priority and the context needed for a correct answer when reorganizing content. Verify the effect through usage records rather than visual similarity.

Should I pad a short prompt to reach the caching threshold?

No: optimize the useful task, not the cache indicator. Unnecessary text can increase total processing and change behavior. Compare the short prompt with the proposed cached version at realistic traffic and reuse levels. The cheapest option may be a short uncached request.

Can cache writes cost more than they save?

Yes, when the write premium and any storage charges are not recovered through enough cheaper reads. The first call can be more expensive even if subsequent calls would save money.

Assume a useful 20,000-token prefix with hypothetical rates of $1 per million ordinary input tokens, $1.25 per million cache-write tokens and $0.10 per million cache-read tokens. One ordinary use costs $0.020, one write costs $0.025 and one read costs $0.002. Assume one write, no expiry, all later requests hitting the same prefix, and no storage fee.

Illustrative prefix-only cost; output and other input excluded
Total usesWithout cachingWith cachingDifference
1: write only$0.020$0.025$0.005 more
2: one write, one hit$0.040$0.027$0.013 saved (32.5%)
10: one write, nine hits$0.200$0.043$0.157 saved (78.5%)
Without caching = N × U
With one write and later hits = W + (N − 1) × R + S
Caching saves money when: N × (U − R) > W − R + S
N = total uses; U = ordinary cost per prefix use
W = write cost; R = read cost; S = total applicable storage cost

The inequality assumes the same prefix, one write and cheaper reads than ordinary input. If entries expire, prefixes change or warm-up calls are added, count every extra write and request. If storage cost changes with retention time, recalculate it for each scenario. The table is not a universal claim that caching pays off after two requests.

Should I choose a longer cache lifetime?

Only when the additional eligible reuse justifies its cost. Compare your actual request intervals with the supported lifetime and rates. Some explicit caching products can charge for retained content: Gemini’s generateContent caching documentation explains explicit caching and billing. Availability differs by model and endpoint; do not apply explicit-cache storage assumptions to all caching modes.

Do not send artificial keep-alive requests solely to improve the hit rate without counting their full cost. A quiet workload may be cheaper with no cache maintenance.

Why is my cache hit rate high but my API bill barely lower?

Caching discounts eligible input, not every dollar spent on the task. New input, output generation, paid tools and retries may still dominate. A hit can also reuse only a small fraction of a large request.

In the ten-use example, prefix cost falls from $0.200 to $0.043: a 78.5% reduction for that component. Suppose output, new input and tools add another $0.800 under both configurations. The total falls from $1.000 to $0.843, a saving of $0.157 or 15.7%. A large input discount and a modest overall saving can both be correct.

Likewise, a lower cost per task can coexist with a higher monthly bill when more users run more tasks. Compare both total spending and cost per active user or comparable execution.

Which cache metric should I look at?

Use several: requests with a read, the share of input tokens read from cache, write-to-read volume and net dollars saved. Define the denominator. For example, cached-input share = cache-read tokens / total input tokens describes input coverage; it is not a direct savings percentage. Review by workflow and model so heavily cached traffic does not hide an expensive poorly cached workflow.

How do I check whether I am calculating caching costs correctly?

Normalize provider usage into mutually exclusive billing categories before multiplying by rates. A nested token count is not always an additional chargeable quantity.

For supported OpenAI Responses usage schemas, input_tokens includes cached input; inspect input_tokens_details.cached_tokens and any supported cache_write_tokens field. The ordinary-input calculation must exclude the applicable read and write categories. Claude Messages instead reports ordinary input_tokens, cache_read_input_tokens and cache_creation_input_tokens separately; sum these to obtain total input. Check the linked provider documentation for your API version and rate categories.

Normalized input cost =
  ordinary tokens × ordinary rate
  + cache-write tokens × applicable write rate
  + cache-read tokens × read rate

Then add output, paid tools and any storage charges.
Divide token quantities by 1,000,000 when rates are per million.

Where a write rate is the full price for those tokens, do not add an ordinary-input charge on top. Split writes by lifetime or other rate category when needed. Keep missing usage or unknown prices visible rather than assigning zero.

What is a useful test before changing production?

  1. Choose a representative workflow and record model, request configuration, task size and acceptance criteria.
  2. Run a controlled repeat with useful stable content, within the applicable lifetime, and inspect actual reads and writes.
  3. Test realistic request intervals and variations, including the changes your application introduces automatically.
  4. Calculate a same-workload uncached counterfactual at the applicable rate, and label it as an estimate. Include warm-up and storage costs in the cached case.
  5. Compare net cost per task, latency and answer quality. Repeat across enough work to avoid drawing conclusions from one unusually cheap run.

If you can run a true uncached comparison, keep the task set and model configuration comparable. Otherwise, distinguish the calculated counterfactual from an observed uncached bill.

What should I change after finding the cause?

Fix the cause that the usage evidence supports. Stabilize a changing reusable prefix, adjust supported cache configuration, or stop writing rarely reused content. If output and tool costs dominate, optimize those instead of chasing a higher cache-hit percentage.

Ganivra can compare recorded execution costs by customer and workflow when you supply that context. Provider usage records remain necessary to diagnose cache behavior; cost attribution alone does not reveal every cache miss. Follow the integration guide for telemetry and the LLM cost-reduction guide for quality checks before rollout.

Frequently asked questions about prompt caching savings

Why isn’t prompt caching saving me money?

Common causes are few actual cache hits, changing prompt prefixes, expired entries, insufficient reuse and write or storage charges. Caching may also be working while output tokens, tools or increased traffic dominate the bill. Compare recorded cache usage and total cost for the same workload.

Why are my cached tokens zero even though I enabled caching?

Check model and endpoint support, minimum eligible length, the reusable prefix, cache configuration and time between requests. The first eligible call may create an entry rather than read one. Verify that you are inspecting the correct usage fields for your API.

Can prompt caching make my API bill higher?

Yes. A cache write can cost more than ordinary input under some pricing schemes, and storage charges may apply to some explicit caches. If content is rarely reused, those costs may exceed read savings. Use your model's actual rates and include all writes, reads and applicable storage.

Why is my cache hit rate high but my bill barely lower?

A request with a small cached prefix still counts as a hit in some dashboards. Measure the share of input tokens read from cache, then compare dollar savings across the whole task. Output generation, new input, tool calls and retries can remain expensive even when cached-input costs fall.

How many times must I reuse a prompt before caching pays off?

It depends on the write price, read discount, expiry and any storage charges. In this guide's hypothetical example, one write costs $0.025, each read costs $0.002 and ordinary input costs $0.020 per use. One write followed by one read costs $0.027 instead of $0.040, provided the second request hits the cache.

Should I make my prompt longer to trigger caching?

Do not add irrelevant text just to reach a threshold. Compare total cost and answer quality using the useful prompt you need. A shorter uncached request can cost less than a padded cached request, especially when reuse is low.

Does prompt caching return an old ChatGPT or Claude answer?

Provider prompt caching reuses processing for eligible input rather than substituting an earlier final answer. Output is still generated. This guide concerns API caching, not consumer subscription billing. An application that stores and returns complete answers is using response caching, which needs separate freshness and access checks.

How do I measure whether prompt caching is worth keeping?

Record ordinary input, cache reads, writes, output and other billed components for a representative workload. Compare against the same input volume at the applicable uncached rate, include warm-up and storage costs, and check quality and latency. Track savings per task as well as the monthly bill.

Is caching reducing the cost of your actual workload?

Use recorded read, write, output and tool costs to compare the complete workflow. Keep the same task volume and success criteria in both scenarios.

Compare workload cost and margin →

The calculator supports workload economics; use provider records and the formula above to diagnose cache behavior.

Method: original illustrative calculations by Ganivra Editorial. Provider documentation reviewed November 21, 2026. Rates in the worked example are hypothetical USD assumptions. No guaranteed cache-hit rate or customer savings are claimed.