Batch processing
Send many chat or embedding requests in one file, and collect the results when the batch finishes.
What is Batch processing?
Batch processing lets you send a large set of AI requests in one file, have them processed asynchronously, and collect all results when the batch finishes. It suits work that does not need an instant answer, such as labelling, summarising and embedding large datasets. Batch work is often priced below real-time calls, depending on the model and provider. Spro's batch processing is planned and not yet available.
Real-time calls are the wrong tool for a backlog of thousands of documents. Batch processing accepts the whole set at once, works through it in the background, and returns every result together.
This page describes planned behaviour. File limits, turnaround times and pricing will be published with the feature.
Key features
What this feature does for your requests.
- Submit many requests in one file
- Results collected when the batch completes
- Chat and embedding requests supported
- Per-request results and usage recorded
How it works
4 steps, from setup to the first request.
- 01
Prepare the file
Write one request per line.
- 02
Submit the batch
Send the file to the gateway.
- 03
Wait for completion
Work runs in the background.
- 04
Collect the results
Download the outputs and the usage for each request.
Is it the right fit?
Specific situations where this feature helps, and where it does not.
Good fit if
- Thousands of documents need labelling, summarising or embedding overnight
- Results are needed by the next day, not in seconds
- You want one submission instead of managing rate limits yourself
Not a good fit if
- A user is waiting for the answer
- A handful of requests that should finish in seconds
- Results must feed a step that runs immediately
Batch processing FAQs
Common questions about Batch processing.
Route every AI request through SPRO.
One OpenAI-compatible API for 350+ models, with intelligent routing, 29 built-in guardrails, observability, and automatic failover.
