---
title: "Cut Costs with Caching"
description: "Send the same stable prompt prefix twice, compare `cacheReadTokens`, and determine whether caching fits the workload."
canonical_url: "https://vercel.com/academy/ai-gateway/prompt-caching"
md_url: "https://vercel.com/academy/ai-gateway/prompt-caching.md"
docset_id: "vercel-academy"
doc_version: "1.0"
last_updated: "2026-08-08T23:27:13.381Z"
content_type: "lesson"
course: "ai-gateway"
course_title: "Using AI Gateway in Production"
prerequisites:  []
---

<agent-instructions>
Vercel Academy — structured learning, not reference docs.
Lessons are sequenced.
Adapt commands to the human's actual environment (OS, package manager, shell, editor) — detect from project context or ask, don't assume.
The lesson shows one path; if the human's project diverges, adapt concepts to their setup.
Preserve the learning goal over literal steps.
Quizzes are pedagogical — engage, don't spoil.
Quiz answers are included for your reference.
</agent-instructions>

# Cut Costs with Caching

# How Do I Cut My Token Bill with Caching?

The Taco Tuesday assistant sends the full menu and operating instructions with every order. Those repeated input tokens can become a significant part of the request cost even when the content has not changed.

Prompt caching lets supported providers reuse that stable prefix instead of charging the full input rate each time.

\*\*Note: Quick Answer\*\*

Set `caching: "auto"` in `providerOptions.gateway`. The Gateway adds cache markers for providers that require them, while other providers detect repeated prefixes themselves. Verify the cache hit in `usage.inputTokenDetails.cacheReadTokens` and compare the two request costs.

## Outcome

Enable automatic caching on the ordering assistant's model and prove, with token counts from two consecutive requests, that the second one read the menu from cache instead of paying full price.

## Fast Track

1. Add `providerOptions: { gateway: { caching: 'auto' } }` to a call with a big stable prefix
2. Run the same request twice
3. Compare `usage.inputTokenDetails.cacheReadTokens`: near zero the first time, nearly the whole menu the second

## Hands-on exercise

Build `scripts/cache-check.ts` with two identical requests and print the token details for each one.

Requirements:

- Use an Anthropic model (`anthropic/claude-sonnet-4.6`), because Anthropic needs the explicit markers, which makes the Gateway's work visible
- Send the full `TRUCK_INSTRUCTIONS` (menu included) as `instructions`, plus a short customer question
- Set `caching: 'auto'` under `providerOptions.gateway`
- After each call, print `inputTokens`, `cacheReadTokens` from `usage.inputTokenDetails`, and the actual cost from `gateway.cost`

Caching requires at least two requests to demonstrate a hit. The first request writes the cache entry and may carry a write premium; the second request can read it at the cached-input rate.

## Try It

```bash
pnpm cache-check
```

```
--- Request 1 ---
Input tokens:      612
Cache read tokens: 0
Cost:              $0.002214

--- Request 2 ---
Input tokens:      612
Cache read tokens: 578
Cost:              $0.000371
```

The second request reports most of the stable prefix as cache-read tokens and shows the lower request cost.

\*\*Warning: When caching costs you money\*\*

Some providers charge more to write a cache entry and less to read it. One-shot prompts can pay the write cost without receiving a later discount. Check the current provider policy and use caching for stable prefixes that repeat.

Two issues you may encounter:

**`cacheReadTokens` is `0` on the second request.** Check the provider's current cache lifetime and confirm that the stable prefix is byte-identical. A timestamp or request-specific value in the instructions prevents a prefix match.

**The second request used another provider.** Cache entries live with a provider, so changing providers can produce a miss. Pin cache-sensitive traffic when consistency matters, as shown in lesson 1.7.

## Commit

```bash
git commit -m "feat(caching): enable automatic prompt caching for the menu prefix"
```

## Done-When

- [ ] `scripts/cache-check.ts` runs two requests and prints both receipts
- [ ] Request 2 shows `cacheReadTokens` covering most of the menu
- [ ] Request 2's actual cost is visibly smaller than request 1's
- [ ] You can say when `caching: 'auto'` would *lose* money (one-shot traffic)

## Solution

```ts filename="scripts/cache-check.ts"
import { generateText } from "ai";
import { TRUCK_INSTRUCTIONS } from "../lib/menu";

async function order(label: string) {
  const result = await generateText({
    model: "anthropic/claude-sonnet-4.6",
    instructions: TRUCK_INSTRUCTIONS,
    prompt: "Which taco should I get if I can't handle spice? Be honest.",
    providerOptions: {
      gateway: { caching: "auto" },
    },
  });

  const { inputTokens = 0, inputTokenDetails } = result.usage;
  const cost = result.finalStep.providerMetadata?.gateway?.cost;

  console.log(`--- ${label} ---`);
  console.log(`Input tokens:      ${inputTokens}`);
  console.log(`Cache read tokens: ${inputTokenDetails?.cacheReadTokens ?? 0}`);
  console.log(`Cost:              $${cost}`);
}

await order("Request 1");
await order("Request 2");
```

The second request's token metadata tells us whether the provider reused the prefix and how that changed the cost.

## Related Questions

- [How is AI Gateway priced?](/ai-gateway/save-money/ai-gateway-pricing)
- [How do I pin routing to a specific provider?](/ai-gateway/stay-reliable/pin-a-provider)
- [How do I see exactly where my money is going?](/ai-gateway/stay-aware/see-your-spend)


---

[Full course index](/academy/llms.txt) · [Sitemap](/academy/sitemap.md)