The repeated instructions on a smaller bill

A small AI task can carry a surprisingly large amount of background material. You ask for a short answer, but the request also includes instructions, examples, tool definitions, and perhaps several pages of reference text. On the next request, much of that material goes along again.

Prompt caching can reduce the cost of that repetition. When a provider supports it, the model service can reuse work already done on an unchanged beginning of a prompt. Those reused input tokens may be billed at a lower rate than fresh input tokens.

§ What a prompt cache actually saves

A token is a small unit of text or other model input. API bills commonly distinguish between input tokens, which the model receives, and output tokens, which it generates. Prompt caching adds another distinction: some input may qualify as cached input.

Imagine an assistant that receives the same handbook before every question. The first request establishes an eligible cache entry. A later request with the same opening material can read that entry instead of processing all of the repeated material from scratch.

This is not a saved-answer feature. The model still handles the new question and generates a response. Caching reuses processing for eligible input; it does not simply return yesterday’s answer.

§ Where the lower bill comes from

Suppose each request contains 10,000 tokens of fixed instructions and reference material, followed by a new 500-token question. Without caching, all 10,500 input tokens are billed at the ordinary input rate.

For an illustration, assume ordinary input costs $1 per million tokens and cached reads cost $0.10 per million tokens. These are hypothetical rates, not Vana API prices.

• Without a cache hit, the input costs $0.0105.
• With all 10,000 fixed tokens read from cache, those tokens cost $0.001.
• The new 500 tokens cost $0.0005, making the input total $0.0015.

That is a substantial reduction in this example’s input charge, but not necessarily the same reduction in the whole bill. Generated output still costs money. Some providers also charge a separate cache-write rate, potentially higher than ordinary input, so establishing a cache has a cost.

§ Keep the repeated material at the beginning

Prompt caches generally depend on a matching prefix: the opening portion of the request. Stable material belongs first, with the changing question or task afterward.

• Keep general instructions, reference documents, and examples in a consistent order.
• Place timestamps, request identifiers, and other changing details after reusable material where practical.
• Avoid rewriting fixed instructions between calls unless they actually need to change.
• Keep tool definitions consistent when the same tools are needed.

A tiny change near the beginning can prevent reuse of everything after it. Similar wording is not enough; exact matching and other provider-specific conditions matter.

§ A cache hit is not guaranteed

Providers differ in their minimum eligible prompt length, cache lifetime, pricing, and configuration. Some cache eligible requests automatically. Others require explicit cache controls or a separate caching workflow.

Entries can expire, and changing the model or routing a request elsewhere may prevent reuse. Short prompts or material used only once may offer little benefit. Repeated, long inputs are usually the more promising place to start.

§ Checking savings through Vana API

Vana API provides one endpoint with OpenAI compatibility, plus support for Anthropic Messages and Gemini native protocols. It bills per real token with no monthly fee. That makes repeated input worth examining, but protocol compatibility alone does not guarantee identical caching behavior across models and routes.

Create a Vana API account, then check the caching support and current pricing for your chosen model and route before estimating savings. Where usage details expose cached reads and writes, inspect those alongside fresh input, output, and total cost.

Compare several similar requests rather than judging the first call. The useful result is not merely a cache hit: it is a lower total charge for the same task, without removing context the model needs.