Observability

Latency tracking

See how long each request takes by model, provider and key, so slow routes show up before users complain.

Duration per requestCompare models and providersFilter by time window

What is Latency tracking?

Latency tracking records how long each request takes, by model, provider, API key and time window. Every request log carries its duration, so slow models and providers are visible and can be compared over time. On Spro you can see latency in the request logs and use it when comparing routes.

A provider can be slow without showing as an error. Users notice response time long before a dashboard shows a problem. Latency tracking puts the duration of every call next to its model and provider, so slow routes are easy to find.

The same data helps when you change routing: compare durations on the same model across providers, or check whether a fallback added time to requests.

Key features

What this feature does for your requests.

  • Duration recorded on every request
  • Compare durations by model and provider
  • Filter by API key and time window
  • Combine with trace IDs to see slow steps in a run

How it works

4 steps, from setup to the first request.

  1. 01

    Send requests

    Each request's duration is recorded automatically.

  2. 02

    Open request logs

    Find the calls you want to check.

  3. 03

    Filter by model or provider

    Narrow the list to one route or key.

  4. 04

    Compare durations

    Check whether one provider or a fallback is slower.

Is it the right fit?

Specific situations where this feature helps, and where it does not.

Good fit if

  • A user-facing feature where response time affects the experience
  • Comparing providers for the same model on speed
  • Checking whether a fallback or retry made requests slower

Not a good fit if

  • Precise timing of each step inside a multi-step agent run. Tracing shows the run
  • A guarantee of response time. Latency is measured, not promised
  • Quality problems. Slow answers and wrong answers need different checks

Latency tracking FAQs

Common questions about Latency tracking.

Route every AI request through SPRO.

One OpenAI-compatible API for 350+ models, with intelligent routing, 29 built-in guardrails, observability, and automatic failover.

Talk to sales