Latency tracking
See how long each request takes by model, provider and key, so slow routes show up before users complain.
What is Latency tracking?
Latency tracking records how long each request takes, by model, provider, API key and time window. Every request log carries its duration, so slow models and providers are visible and can be compared over time. On Spro you can see latency in the request logs and use it when comparing routes.
A provider can be slow without showing as an error. Users notice response time long before a dashboard shows a problem. Latency tracking puts the duration of every call next to its model and provider, so slow routes are easy to find.
The same data helps when you change routing: compare durations on the same model across providers, or check whether a fallback added time to requests.
Key features
What this feature does for your requests.
- Duration recorded on every request
- Compare durations by model and provider
- Filter by API key and time window
- Combine with trace IDs to see slow steps in a run
How it works
4 steps, from setup to the first request.
- 01
Send requests
Each request's duration is recorded automatically.
- 02
Open request logs
Find the calls you want to check.
- 03
Filter by model or provider
Narrow the list to one route or key.
- 04
Compare durations
Check whether one provider or a fallback is slower.
Is it the right fit?
Specific situations where this feature helps, and where it does not.
Good fit if
- A user-facing feature where response time affects the experience
- Comparing providers for the same model on speed
- Checking whether a fallback or retry made requests slower
Not a good fit if
- Precise timing of each step inside a multi-step agent run. Tracing shows the run
- A guarantee of response time. Latency is measured, not promised
- Quality problems. Slow answers and wrong answers need different checks
Latency tracking FAQs
Common questions about Latency tracking.
Route every AI request through SPRO.
One OpenAI-compatible API for 350+ models, with intelligent routing, 29 built-in guardrails, observability, and automatic failover.
