Prompt compression
Shorten long prompts at the gateway so you pay for fewer input tokens.
What is Prompt compression?
Prompt compression is a gateway feature that shortens a long prompt before it reaches the model, so fewer input tokens are billed. The aim is the same answer for less input cost. It matters most for long documents, long chat histories and retrieved context.
Long prompts are often mostly filler: repeated instructions, old chat turns and long retrieved passages that the model does not need in full. Compression removes that text, so the input bill goes down.
This page describes planned behaviour. Compression changes the text the model sees, so check answer quality on your own workloads before you turn it on for production traffic.
Key features
What this feature does for your requests.
- Shortens long inputs before the provider call
- Targets repeated and low-value text first
- Savings shown as fewer input tokens in usage logs
- Turned on per request or per workspace
How it works
4 steps, from setup to the first request.
- 01
Send a long prompt
Use the same endpoint and model you already use.
- 02
Compression runs at the gateway
Repeated and low-value text is shortened before the call.
- 03
The model answers
It receives the shorter prompt.
- 04
Check the savings
Usage logs show fewer input tokens and the lower cost.
Is it the right fit?
Specific situations where this feature helps, and where it does not.
Good fit if
- Long retrieved documents or chat histories dominate the input tokens
- The prompt contains repeated or low-value text that costs money
- You can test answer quality with and without compression
Not a good fit if
- Short prompts with little to remove
- Exact wording matters, such as legal text or code the model must copy
- Output quality cannot be checked after compression
Prompt compression FAQs
Common questions about Prompt compression.
Route every AI request through SPRO.
One OpenAI-compatible API for 350+ models, with intelligent routing, 29 built-in guardrails, observability, and automatic failover.
