HridaSpro Docs

Advanced

Stream Cancellation

What happens to usage and billing when a client disconnects from a stream mid-generation.

When a client disconnects from a streaming request — user closes the tab, your timeout fires, you break out of the loop — the provider has usually already delivered most of the response. Spro makes sure that work still shows up in your usage logs, with a per-provider policy for how the tokens are accounted.

The guarantee

A cancelled stream is never logged as zero tokens.

This matters more than it sounds. Providers deliver usage stats in the final chunk of a stream — the chunk that never arrives when you disconnect. A naive gateway sees the client leave, gives up, and logs nothing — while the provider still bills the prompt (and often the full generation). Spro's stream pipeline runs decoupled from your connection, so disconnects are detected reliably and usage is always recorded under the request's trace ID.

How it works

When a disconnect is observed mid-stream, Spro applies the stream cancellation strategy configured for the serving provider:

StrategyWhat Spro doesWhen it's used
estimate (default)Estimates usage: output tokens from the text actually streamed (~1 token / 3.5 chars of content), input tokens from the request body — the prompt is billed on every cancelled stream.Most providers.
exactKeeps reading the upstream stream to its end — the provider finishes generating, and the real usage chunk arrives and is logged exactly.Providers that bill the full completion regardless of when you stop reading.
zeroLogs zero tokens.Providers that confirmed they don't bill cancelled streams.

You don't configure this — it's set per provider in Spro's catalog based on how that provider actually behaves. The result is the same either way: your usage logs reflect what you'll actually be billed.

A couple of details worth knowing:

  • Usage already reported = no guessing. If the disconnect lands after the provider sent its usage chunk, that exact usage is logged — no strategy is applied.
  • exact drains produce a normal completion record. If Spro keeps reading and the upstream finishes, the request is logged as a completed stream (with exact tokens), not a cancelled one.
  • Disconnects are marked. A cancelled request is visible in your usage logs with the disconnect as its end reason, distinguishable from streams that ended normally.

Examples

Disconnect mid-stream (Python)

from openai import OpenAI
client = OpenAI(    base_url="https://spro.hridaai.com/v1",    api_key="spro-xxxxxxxxxxxxxxxx",)
response = client.chat.completions.create(    model="gpt-oss-20b",    messages=[{"role": "user", "content": "Write a long story"}],    stream=True,    extra_headers={"x-spro-trace-id": "run_9f2c1a"},)
for chunk in response:    if chunk.choices and chunk.choices[0].delta.content == "END":        break  # client stops reading here...
# ...but the request still appears in your Spro usage logs —# with the tokens served so far (estimate strategy) or the# exact tokens (exact strategy).

Client abort (JavaScript)

const controller = new AbortController();setTimeout(() => controller.abort(), 3000); // cancel after 3s
const response = await fetch("https://spro.hridaai.com/v1/chat/completions", {  method: "POST",  headers: {    "Content-Type": "application/json",    Authorization: "Bearer spro-xxxxxxxxxxxxxxxx",  },  signal: controller.signal,  body: JSON.stringify({    model: "gpt-oss-20b",    stream: true,    messages: [{ role: "user", content: "Write a long story" }],  }),});
// Aborting closes the connection — Spro accounts for the// cancelled stream server-side, exactly as above.

Timeout on the SDK

# A read timeout raises in your code but the same server-side# accounting applies — the disconnect is just observed earlier.client = OpenAI(    base_url="https://spro.hridaai.com/v1",    api_key="spro-xxxxxxxxxxxxxxxx",    timeout=5.0,)

What this means for your app

  • Aggressive timeouts are safe. Cancelling a slow stream doesn't create free or phantom requests — cost tracking stays honest.
  • Prefer short generations over disconnects. estimate is deliberately conservative, but exact is exact — using max_tokens / max_completion_tokens to bound output avoids estimation entirely.
  • Trace it anyway. Cancelled streams land in your logs under the same trace ID as the rest of the run, so an aborted agent step is visible where it happened.

What's Next