Provider routing
Send the same model to the best available provider, and move traffic when one fails.
What is Provider routing?
Provider routing is how an AI gateway picks which cloud provider serves a request when several providers host the same model. Spro sends each request to the highest-priority provider, fails over to the next one when a provider returns an error, and skips a failing provider for 60 seconds. You can pin a request to one provider when you need predictable routing.
Many models are served by more than one provider. The same open-weight model can run on several hosts, each with its own price, speed and uptime. Provider routing lets you use that choice without changing your code: you send one model name and Spro picks the provider.
Spro offers two routing strategies per workspace. Failover, the default, tries providers in priority order until one answers. Priority always uses the top provider and never switches. Each response reports which provider served it in the x-spro-resolved-model header.
Key features
What this feature does for your requests.
- Failover across every provider that serves the model, in priority order
- Priority mode for workloads that must go to one provider
- A failing provider is skipped for 60 seconds, so requests do not wait on it
- Pin a provider in the model name, such as openai/gpt-oss-20b
- Every response reports the model and provider that served it
How it works
4 steps, from setup to the first request.
- 01
Send a plain model name
Use the model slug, for example gpt-oss-20b. Spro can use any provider that serves it.
- 02
Spro ranks the providers
Providers are tried in priority order for the workspace's routing strategy.
- 03
Failover on errors
If a provider returns an error, Spro moves to the next one and records the reason.
- 04
Read what served it
The x-spro-resolved-model and x-spro-fallback-from headers show the result.
Is it the right fit?
Specific situations where this feature helps, and where it does not.
Good fit if
- Your app depends on one model that several providers host, and an outage would hurt users
- You want to compare providers for the same model on price or speed
- A workload has to keep answering when one provider has an incident
Not a good fit if
- You need all traffic on one provider for contract or data-location reasons. Pinning works, but then that provider's outage is your outage
- Your prompts rely on behaviour that differs between hosts of the same model
Provider routing FAQs
Common questions about Provider routing.
Route every AI request through SPRO.
One OpenAI-compatible API for 350+ models, with intelligent routing, 29 built-in guardrails, observability, and automatic failover.
