aq.dev / guides / prompt-caching-for-coding-agents-explained

Prompt Caching for Coding Agents, Explained

Prompt caching lets a model provider reuse the unchanged beginning of a request, called the prefix, so a coding agent pays the cached-input rate for history it has already processed. In Claude Code, changing the model, effort level, tools, or fast/standard speed can make the next turn read part or all of the conversation uncached, with documented exceptions for some models, authentication methods and deferred tools. The cache lives with the model provider, and a miss usually means one slower, more expensive turn while the new prefix is written.

AQ gives teams shared live coding-agent workspaces with isolated worktrees, real CLIs, and shared terminal, editor and preview access on team-controlled or AQ-managed machines. AQ does not change the provider's cache, billing, entitlement or retention rules described in this guide.

Why coding agents depend on caching

A model API is stateless: it remembers nothing between requests, so an agent harness sends the full history on every turn, and an agent task is rarely one request. A single "fix the failing test" task might be 30 model requests; at 150,000 tokens of accumulated context, that is about 4.5 million input tokens without caching. With caching, each request bills the unchanged prefix at the cached rate and only the newly appended content at full price.

Agent sessions are cache-friendly when the conversation grows by appending to the end. The match must be exact, so one changed value early in the request can reprocess everything after it at full price. A cache write is the request that stores a new prefix, and some vendors charge extra for it. A cache read, or hit, is a later request reusing that entry at the discounted rate.

What causes a Claude Code cache miss

Anthropic's current Claude Code prompt-caching documentation orders a request into the system prompt, project context and conversation. A conversation change can leave the earlier layers cached; a system-prompt change invalidates everything after it. There is no per-file cache that can rescue the rest of an edited prefix.

  • Exact-prefix matching. Cache hits require identical content through the cached breakpoint. On the API, entries are written at breakpoints and reads look backward for a prior write; changing tool definitions invalidates the tools, system and message layers, while a message-layer change has a smaller blast radius. The API prompt-caching reference also documents a 20-block lookback window per breakpoint and up to four explicit breakpoints.
  • Switching models. Each model has its own cache. A /model switch makes the next request read the whole conversation uncached even when every byte is identical. The same applies to an opusplan plan-mode switch, a skill or command that selects another model for one turn, and automatic model fallback. Claude Code warns before a model switch only while the cache is warm and the new model did not produce the last response, according to the model-switch rules.
  • Changing effort. On most models, each effort level has its own cache, so /effort causes a full read on the next request. With API-key or Claude-subscription authentication, effort changes preserve the cache by default on Fable 5.1 and later, Opus 5.5, Sonnet 5.5 and Haiku 5.5. That exception does not apply on Amazon Bedrock, Google Cloud's Agent Platform, a Claude apps gateway, a HIPAA-configured organization, or with CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS set. These boundaries come from Anthropic's effort-change documentation; do not generalize them to other models or providers.
  • Changing speed. Fast and standard modes keep separate caches. The first fast-mode request in a conversation reads the whole conversation uncached at fast-mode rates. Every later toggle rereads whatever the selected speed has not already cached, including Claude Code's automatic fallback to standard at a fast-mode rate limit and its return to fast after cooldown. Enable fast mode at a session boundary when that trade-off is appropriate; Anthropic documents lower latency at higher token prices, not better task quality. See fast mode and the speed-cache rules.
  • Connecting or removing MCP tools. With deferred tool search, the default on supported models, Claude Code keeps the tool list from the conversation's first request. An MCP server connecting or dropping mid-session therefore leaves the cached prefix intact, and a server that connects later supplies deferred definitions on demand. With tools loaded upfront, adding a definition or intentionally removing one invalidates the cache. An unexpected server dropout keeps the old definitions and the cache; a reconnect can add WaitForMcpServers and cause one miss. Editing MCP config alone takes effect only after restart. The full decision table is in Anthropic's MCP cache documentation, and the MCP guide covers the server setup itself.
  • Plugins and deny rules have different cases. Plugin skills, commands, agents, hooks, monitors and themes append content after the cached prefix, so enabling or disabling them preserves it. A plugin that supplies MCP servers follows the MCP rules above. With tool search active, denying an entire tool leaves the definitions unchanged and preserves the cache; without tool search, adding or removing that deny rule invalidates it. Scoped rules such as Bash(rm *), and allow and ask rules, are checked when a call is attempted and leave the prefix intact, per the plugin and permission cases.
  • Compaction. Compaction replaces the conversation with a summary, so the conversation cache changes by design. The system prompt and unchanged project context can stay cached. While the cache is warm, the separate summarization request can read the old prefix; after a cold resume, that request processes the history uncached. The next turn then caches the shorter summary. Running /compact at a natural break lets you choose when this work happens, as explained in the compaction documentation. For the underlying window, see context windows for coding agents.
  • Image and PDF removal. When a request would exceed the applicable count or total-size limit, Claude Code removes a batch of the oldest images and PDFs. The next request reprocesses the conversation from the earliest removed item onward, so this can be a partial rather than a full miss. Batching means one slower turn per removal batch, per Anthropic's image-removal documentation.
  • Upgrading Claude Code. A new version typically changes the system prompt or tool definitions, so the first conversation after an upgrade rebuilds the cache from the top. Auto-update applies the new version on the next launch rather than mid-session. Resuming an older conversation reuses only layers whose content and cache lifetime still match, according to the upgrade and resume documentation.
  • Gateway handling of cache_control. If a gateway forwards the markers unchanged, caching behaves as it does at the provider endpoint. If it rejects the marked request with a 400 naming cache_control, Claude Code retries with the marker moved to the last conversation message: the mid-conversation system block is uncached, while the conversation can stay cached. If the gateway silently strips the markers and returns success, the entire conversation bills as uncached input on every turn. Converting block-form system content to a plain string has the same effect. These behaviors are documented under where the cache lives.

For direct API integrations, preserved thinking adds a related constraint. On Fable 5.2, Anthropic documents the prefix check on every account: editing an earlier system prompt, tool set or message can return a 400 when later thinking blocks are replayed, rather than merely causing a cache miss. Server-side compaction and context editing have explicit handling for this case in the preserved-thinking reference.

Fast mode: the price and the second cache

Anthropic says fast mode uses the same model with an API configuration that can produce up to 2.5 times as many output tokens per second, with the same quality and capabilities. That is Anthropic's latency claim, not an independent measurement of coding speed or task outcomes. Fast mode is available on Fable 5.2 and Opus 5.5; Opus 5 and Opus 4.8 are in research preview.

ModelStandard input / output per MTokFast input / output per MTokStatus
Fable 5.2$8 / $40$16 / $80Available
Opus 5.5$4 / $20$8 / $40Available
Opus 5$5 / $25$10 / $50Research preview
Opus 4.8$5 / $25$10 / $50Research preview; removal November 30, 2026

These are Anthropic's current standard and fast-mode rates, checked November 12, 2026. Prompt-caching multipliers apply on top of fast-mode prices. Anthropic's fast-mode guide says Opus 4.8 fast mode was deprecated on October 30, 2026 and will be removed on November 30, 2026; after removal, requests that still select Opus 4.8 with fast mode will be rejected. Switch to Opus 5.5 or turn fast mode off before that date.

On Pro, Max, Ultra, Team and Enterprise subscriptions, fast mode spends usage credits even when included plan usage remains. It is not part of the subscription allowance. Team and Enterprise organizations also need an Owner to enable it, while Console organizations need paid usage and, for Opus 5 or Opus 4.8, provisioned access. At a fast-mode rate limit, Claude Code falls back to standard speed and pricing, then returns to fast after cooldown; exhausted usage credits instead cause rejected fast requests to be retried at standard speed. Each automatic speed change is also a cache toggle, so the first fast turn and later switches can carry uncached input at the selected speed's rates. The requirements and fallback documentation has the account-specific details.

Provider prices, lifetimes and defaults

The mechanisms above are Claude Code's rules, not a universal caching contract. This table keeps a small provider comparison using each vendor's current documentation, checked November 12, 2026. Prices are USD per million tokens unless stated otherwise.

Provider and modelOrdinary inputCached inputWrites or storageLifetime and minimum
Claude Sonnet 5.5$2$0.10 (0.05x)$2.50 for 5m; $4 for 1h5m or 1h, refreshed by reads; 512-token minimum
Claude Opus 5.5$4$0.20 (0.05x)$5 for 5m; $8 for 1h5m or 1h, refreshed by reads; 512-token minimum
Claude Fable 5.2$8$0.20 (0.025x)$10 for 5m; $16 for 1h5m or 1h, refreshed by reads; 512-token minimum
OpenAI GPT-6.1 Sol$2$0.10 (0.05x)$2.50 for 30m; $5 for 12hMinimum 30m or 12h; 1,024 visible input tokens
OpenAI GPT-5.5$5$0.50 (0.1x)No cache-write charge24h retention, typically around 30 minutes and up to 24 hours; minimum varies by request
Gemini 3.1 Pro Preview, prompts up to 200k$2$0.20Explicit cache storage: $4.50 per million tokens per hourImplicit caching is on by default; 4,096-token minimum

Anthropic supports automatic caching through a top-level cache_control field and up to four explicit breakpoints on the Messages API. Claude Code manages those markers unless caching is disabled. Its default TTL is one hour for the main conversation on a Claude subscription within included plan usage, and five minutes with usage credits, an API key or a cloud provider. Other request buckets default to five minutes apart from server-controlled helpers. Every cache read refreshes the applicable TTL at no charge, measured from the start of that request.

OpenAI enables caching by default on supported models. GPT-5.6 and later offer 30-minute writes at 1.25x ordinary input and 12-hour writes at 2.5x, with 0.1x reads on most models and 0.05x on GPT-6.1 Sol and Terra. Earlier models use different retention and minimum rules; the OpenAI prompt-caching guide is the source for those distinctions. GPT-6 models can preserve an earlier prefix during an effort change by appending a configuration_update, which is a different mechanism from Claude's model and authentication exception.

Google enables implicit caching by default on Gemini 2.5 and newer models. The current context-caching guide lists 4,096 tokens for Gemini 3.1 Pro Preview and 2,048 for Gemini 2.5 Pro and Flash, and recommends similar prefixes sent close together. Explicit caching remains available for supported Gemini 3 and 2.5 models through generateContent, with storage billed hourly; the Interactions API and Gemini 4 models use implicit caching only.

A reproducible Claude Code cache check

You can inspect the behavior with a fabricated repository and no credentials in its files. Use your normal Claude sign-in, and do not paste an API key or session token into the repository or a prompt. Anthropic added the Prompt cache (main) summary in Claude Code v2.1.251 and its likely-cause text in v2.1.260; use a current release. The following setup creates harmless source, a local MCP server and a status line that prints current_usage.cache_read_input_tokens and current_usage.cache_creation_input_tokens.

mkdir claude-cache-lab
cd claude-cache-lab
git init
mkdir -p src .claude

python3 - <<'PY'
from pathlib import Path
Path("README.md").write_text(
    "# Cache lab\n\n" +
    "\n".join(
        f"{n}: The sample repository contains only fabricated notes for a cache check."
        for n in range(1, 241)
    ) + "\n"
)
Path("src/calc.js").write_text(
    "export function add(left, right) { return left + right; }\n"
)
PY

cat > src/fake-mcp-server.mjs <<'JS'
import readline from "node:readline";
const rl = readline.createInterface({ input: process.stdin });
rl.on("line", (line) => {
  const request = JSON.parse(line);
  if (request.id === undefined) return;
  let result = {};
  if (request.method === "initialize") {
    result = {
      protocolVersion: request.params?.protocolVersion ?? "2025-06-18",
      capabilities: { tools: {} },
      serverInfo: { name: "cache-lab", version: "1.0.0" }
    };
  } else if (request.method === "tools/list") {
    result = { tools: [{
      name: "lab_note",
      description: "Return a harmless note for a cache check",
      inputSchema: { type: "object", properties: {}, additionalProperties: false }
    }] };
  } else if (request.method === "tools/call") {
    result = { content: [{ type: "text", text: "Fabricated MCP note" }] };
  }
  process.stdout.write(JSON.stringify({ jsonrpc: "2.0", id: request.id, result }) + "\n");
});
JS

cat > .claude/statusline-usage.mjs <<'JS'
let input = "";
process.stdin.on("data", (chunk) => { input += chunk; });
process.stdin.on("end", () => {
  const usage = JSON.parse(input).current_usage ?? {};
  console.log(
    "cache read " + (usage.cache_read_input_tokens ?? 0) +
    " | cache write " + (usage.cache_creation_input_tokens ?? 0)
  );
});
JS

cat > .claude/settings.json <<'JSON'
{
  "statusLine": {
    "type": "command",
    "command": "node .claude/statusline-usage.mjs"
  }
}
JSON

claude mcp add --transport stdio cache-lab -- node "$PWD/src/fake-mcp-server.mjs"
claude

Run two ordinary prompts first: Read README.md and src/calc.js, then explain the function in one sentence. followed by Using the same files, name one input and the returned value. Run /usage after each. The Session block's Prompt cache (main) line reports the per-session hit ratio, miss count, current warmth and, when Claude Code can identify it, the likely cause of the last miss. Record the values after every scenario below rather than expecting a particular number. The documented expectation is a guide to what changed, not output captured from this fabricated repository.

ScenarioAction in a fresh or cleared sessionDocumented behavior to compare with /usage
Normal turnsRun the two baseline prompts above within the cache TTL.The second turn can read the unchanged prefix. Warmth depends on the TTL and request timing.
Model changeWarm the cache, choose another model with /model, send a short follow-up, then inspect /usage.The next request reads the whole conversation uncached because caches are model-specific.
Effort changeWarm a fresh session, change level with /effort, send a follow-up and inspect the likely cause.Most models reread the conversation. Fable 5.1+ and Opus/Sonnet/Haiku 5.5 preserve it only with API-key or Claude-subscription auth.
MCP with deferred toolsOn a supported model with tool search at its default, warm a session, disable and later re-enable cache-lab through /mcp, sending a short prompt after each change.Deferred tool search keeps the initial tool list, so these changes preserve the prefix. Repeat with upfront tools to see intentional additions or removals invalidate it.
Fast/standard toggleOnly with fast mode and usage credits intentionally enabled, warm a standard-mode session on Opus 5.5, run /fast on, send a follow-up, then run /fast off and send another.The first fast request reads the whole conversation uncached at fast rates. Returning to standard reuses only what standard cached and is still warm.

For non-interactive confirmation, Anthropic's cache-check instructions use claude -p "hello" --output-format json. Its usage.cache_creation.ephemeral_1h_input_tokens and ephemeral_5m_input_tokens fields show which TTL the main conversation wrote. The status line above exposes the per-turn current_usage read and creation counts; a high read-to-creation ratio across ordinary turns indicates reuse, while repeated creation points to a changing prefix or one of the speed, model and lifetime boundaries above.

Caching, rate limits and cache scope

Anthropic's API rate limits count ordinary input and cache-write tokens toward input tokens per minute, but exclude cache-read tokens. OpenAI's prompt-caching FAQ says cached tokens still count toward its tokens-per-minute limits. On a Claude subscription, cache behavior also affects how quickly included usage is consumed, but it does not create extra allowance; Claude Code usage limits explains those windows, and coding agent token costs covers the budget arithmetic.

A cache is provider-side. Claude Code's effective cache scope is one machine and directory because the working directory, platform, shell, OS and memory paths enter the prefix; matching sessions in the same directory can share entries. At the API layer, caches are isolated between organizations and, on the Claude API, Claude Platform on AWS and Microsoft Foundry, between workspaces. Sharing also requires a matching model, speed and prefix. These boundaries come from the Claude Code cache-scope documentation and API storage reference.

Where AQ fits

AQ is the multiplayer coding harness where engineering teams run AI coding agents like Claude Code and Codex together: shared live terminals, a code editor, and app previews, on team-controlled or AQ-managed machines. Agents run as real CLIs in persistent sessions inside isolated git worktrees. Teammates with access can open the same live workspace, watch the terminal, use the editor and review the preview; terminal control requires the owner's approval.

Each person connects their own agent subscriptions, and AQ adds no markup to that usage. The cache stays between the CLI and its provider: AQ does not move it, extend its lifetime, combine it across organizations, change fast-mode entitlement or alter the bill.

On the current AQ pricing page, Free is for one person running agents on their own Mac with the AQ Mac app and includes no AQ credits. AQ Pro is $50 per user per month in early access (standard $200), with team-controlled or AQ-managed machines and $10 of AQ credits per seat each month. Your own Claude, OpenAI, Cursor and other agent subscriptions carry no AQ markup.

Frequently asked questions

Why is my Claude Code session expensive after a break?

The cached prefix may have expired. Anthropic offers five-minute and one-hour TTLs, and each cache read refreshes the timer from the start of that request. Claude Code defaults the main conversation to one hour on a Claude subscription within included plan usage, and five minutes with usage credits, an API key or a cloud provider; other request buckets generally use five minutes. The first turn after a longer gap reprocesses the conversation and writes a fresh entry. See Anthropic's cache-lifetime rules, current as of November 12, 2026.

Does switching models or effort mid-session cost extra?

Usually yes for the next turn. Model caches are separate, and most models also separate caches by effort, so the whole conversation is read uncached once. With API-key or Claude-subscription auth, Fable 5.1 and later plus Opus, Sonnet and Haiku 5.5 preserve the cache across effort changes; other providers and configurations follow the ordinary invalidation rule. Switching back can reuse an earlier entry only if it is still within its TTL. These are Anthropic's documented model and effort rules.

What does Claude Code fast mode cost, and is it included in a subscription?

Fast mode costs $16/$80 per MTok input/output on Fable 5.2, $8/$40 on Opus 5.5, and $10/$50 on Opus 5 and Opus 4.8. Standard rates are respectively $8/$40, $4/$20 and $5/$25. On subscriptions it draws usage credits even when included plan usage remains; Team and Enterprise also require Owner enablement. The first fast request in a conversation reads the full context uncached at fast rates, so enabling at a session boundary avoids carrying a long history into that charge. Sources: Anthropic's pricing and fast-mode requirements, checked November 12, 2026.

Does fast mode improve coding answers?

Anthropic says fast mode uses the same model with the same quality and capabilities, at up to 2.5 times the output-token speed and higher per-token prices. That is a vendor latency claim, not evidence of better coding outcomes. Effort is a separate control: lowering it can reduce thinking time and can reduce quality on complex tasks. See Anthropic's fast mode versus effort comparison.

What happens at a fast-mode rate limit or when credits run out?

At a fast-mode rate limit, Claude Code falls back to standard speed and pricing, then automatically returns to fast after cooldown. When usage credits run out, rejected fast requests are retried at standard speed and pricing. Because fast and standard have separate caches, either transition can reread whatever the selected speed has not cached. Anthropic documents the fallback behavior. Opus 4.8 fast mode is a separate deadline: it is scheduled for removal on November 30, 2026.

Do I need to configure prompt caching in Claude Code, Codex, or Gemini CLI?

For normal interactive use, the supported CLIs handle caching for you. Claude Code manages cache markers unless disabled, OpenAI enables prompt caching by default on supported models, and Google enables implicit caching by default on Gemini 2.5 and newer models. Direct API use differs: Anthropic's Messages API uses cache_control, OpenAI supports implicit and explicit breakpoints on newer models, and Google's explicit cache support depends on model and endpoint. Check the current Claude, OpenAI and Google guides for your integration.

Does prompt caching help with rate limits and usage limits?

On Anthropic's API, cache reads do not count toward input-tokens-per-minute limits, although ordinary input and cache writes do. On OpenAI's API, cached tokens still count toward tokens-per-minute limits. A high hit rate can therefore increase Anthropic API throughput, while both providers charge less for cached input. Subscription windows are a different control: caching can reduce consumption but does not raise the plan's cap or fast-mode entitlement. Sources: Anthropic rate limits and OpenAI prompt caching.