Cost controlComing soon

Batch processing

Send many chat or embedding requests in one file, and collect the results when the batch finishes.

Coming soonOne file, many requestsAsynchronous results

What is Batch processing?

Batch processing lets you send a large set of AI requests in one file, have them processed asynchronously, and collect all results when the batch finishes. It suits work that does not need an instant answer, such as labelling, summarising and embedding large datasets. Batch work is often priced below real-time calls, depending on the model and provider. Spro's batch processing is planned and not yet available.

Real-time calls are the wrong tool for a backlog of thousands of documents. Batch processing accepts the whole set at once, works through it in the background, and returns every result together.

This page describes planned behaviour. File limits, turnaround times and pricing will be published with the feature.

Key features

What this feature does for your requests.

  • Submit many requests in one file
  • Results collected when the batch completes
  • Chat and embedding requests supported
  • Per-request results and usage recorded

How it works

4 steps, from setup to the first request.

  1. 01

    Prepare the file

    Write one request per line.

  2. 02

    Submit the batch

    Send the file to the gateway.

  3. 03

    Wait for completion

    Work runs in the background.

  4. 04

    Collect the results

    Download the outputs and the usage for each request.

Is it the right fit?

Specific situations where this feature helps, and where it does not.

Good fit if

  • Thousands of documents need labelling, summarising or embedding overnight
  • Results are needed by the next day, not in seconds
  • You want one submission instead of managing rate limits yourself

Not a good fit if

  • A user is waiting for the answer
  • A handful of requests that should finish in seconds
  • Results must feed a step that runs immediately

Batch processing FAQs

Common questions about Batch processing.

Route every AI request through SPRO.

One OpenAI-compatible API for 350+ models, with intelligent routing, 29 built-in guardrails, observability, and automatic failover.

Talk to sales