Cost controlComing soon

Prompt compression

Shorten long prompts at the gateway so you pay for fewer input tokens.

Coming soonFewer input tokensChecked against quality

What is Prompt compression?

Prompt compression is a gateway feature that shortens a long prompt before it reaches the model, so fewer input tokens are billed. The aim is the same answer for less input cost. It matters most for long documents, long chat histories and retrieved context.

Long prompts are often mostly filler: repeated instructions, old chat turns and long retrieved passages that the model does not need in full. Compression removes that text, so the input bill goes down.

This page describes planned behaviour. Compression changes the text the model sees, so check answer quality on your own workloads before you turn it on for production traffic.

Key features

What this feature does for your requests.

  • Shortens long inputs before the provider call
  • Targets repeated and low-value text first
  • Savings shown as fewer input tokens in usage logs
  • Turned on per request or per workspace

How it works

4 steps, from setup to the first request.

  1. 01

    Send a long prompt

    Use the same endpoint and model you already use.

  2. 02

    Compression runs at the gateway

    Repeated and low-value text is shortened before the call.

  3. 03

    The model answers

    It receives the shorter prompt.

  4. 04

    Check the savings

    Usage logs show fewer input tokens and the lower cost.

Is it the right fit?

Specific situations where this feature helps, and where it does not.

Good fit if

  • Long retrieved documents or chat histories dominate the input tokens
  • The prompt contains repeated or low-value text that costs money
  • You can test answer quality with and without compression

Not a good fit if

  • Short prompts with little to remove
  • Exact wording matters, such as legal text or code the model must copy
  • Output quality cannot be checked after compression

Prompt compression FAQs

Common questions about Prompt compression.

Route every AI request through SPRO.

One OpenAI-compatible API for 350+ models, with intelligent routing, 29 built-in guardrails, observability, and automatic failover.

Talk to sales