Performance

Prompt caching

Reuse the provider's processed prompt prefix to cut time to first token and input cost.

Provider-sidePassed through untouchedWorks with response caching

What is Prompt caching?

Prompt caching is a provider feature that reuses the already-processed start of a prompt, so repeated long prompts cost less and begin faster. Spro passes prompt-caching fields to the provider unchanged and shows cached input tokens in your usage logs. It is different from Spro's response caching, which skips the model entirely.

Long system prompts, tool definitions and reference documents often repeat across requests. A provider can keep the processed prefix and skip re-reading it, which lowers input cost and time to first token. The model still writes a new answer.

Spro does not replace the provider's prompt cache. It forwards the cache fields you send, so they reach the provider as written. Cached input tokens appear in the usage log, so you can see the savings on each request.

Key features

What this feature does for your requests.

  • Pass-through of provider cache fields such as cache_control
  • Cached input tokens shown in usage logs at the provider's reduced rate
  • Lower time to first token on long, repeated prompts
  • Combines with Spro response caching on the same account
  • Works with providers that support prompt caching

How it works

4 steps, from setup to the first request.

  1. 01

    Put stable content first

    Keep the system prompt and reference documents at the start of the prompt.

  2. 02

    Mark the cache point

    Add the provider's cache fields to the request body.

  3. 03

    Spro forwards it

    The fields reach the provider untouched.

  4. 04

    Check usage

    The usage log shows cached input tokens and their discounted cost.

Is it the right fit?

Specific situations where this feature helps, and where it does not.

Good fit if

  • Long system prompts or tool definitions sent on every request
  • RAG apps that resend the same large documents across turns
  • Agents with a long, fixed set of instructions

Not a good fit if

  • Short prompts, where the repeated part is a small share of the input
  • Prompts that change at the start on every call, so nothing repeats
  • Expecting it to shorten the model's output or speed up its writing

Prompt caching FAQs

Common questions about Prompt caching.

Route every AI request through SPRO.

One OpenAI-compatible API for 350+ models, with intelligent routing, 29 built-in guardrails, observability, and automatic failover.

Talk to sales