Skip to main content
A batch is one job holding many requests to one model. Opper runs it on the provider’s batch API, which costs about half the synchronous price and returns the results within 24 hours. You send the requests inline in one JSON body, poll the batch, and read the results when it ends.
Batch runs on OpenAI, Anthropic and DeepInfra models in the US, and on Azure OpenAI, Google’s Gemini API and Mistral (global). In the EU it runs on Google Vertex AI for Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5, Gemini 3.8 Flash and Gemini 3.5 Flash Lite, on AWS Bedrock in Stockholm for Claude Haiku 4.5 and Claude Opus 5, on Sference, and on Scaleway in Paris. See Batch in the EU.

Create a batch

Each body is the normal request for that endpoint and goes to the provider as it is. Opper sets model in each body for you, so you can leave it out. For a Claude batch, each body is an Anthropic Messages request:
For a Gemini batch, each body is a Gemini generateContent request:
A body in another format than the model’s batch endpoint takes (a chat completions body for a Claude model, for example) answers 400 naming the endpoints the model takes. stream: true also answers 400. The response is 202 with the batch object, in status validating. Every custom_id and body is checked before anything is accepted: if one line is malformed, the whole batch is refused. A body the provider rejects later comes back as that line’s error in the results.

Poll the batch

The batch object carries the fields of OpenAI’s batch object plus OpenRouter’s, so a client written for either can read it:
There is no cancel endpoint. Most batches finish in minutes; poll every 30 to 60 seconds, or get a webhook instead.

Get a webhook instead of polling

Set callback_url on the batch, and Opper sends one signed POST there when it ends: batch.completed, batch.failed, batch.cancelled or batch.expired. Its data is the batch object above, without the results, and its status_url is GET /v3/batches/{id}, which has them. batch.completed can still hold failed requests, so check data.request_counts.failed.
A batch without callback_url uses the project’s default webhook URL when one is set (Settings > Webhooks). Signing, retries and URL rules are the same as for video and image jobs: see Webhooks. GET /v3/batches/{id} shows the delivery under webhook.

Read the results

Once the batch has ended (completed, expired or cancelled), GET /v3/batches/{id} includes results inline. Each line is OpenAI’s batch result line, with the provider’s response body as it is:
Results are not in request order: match them by custom_id. For a large batch, page through GET /v3/batches/{id}/results instead of reading them all in one response:
limit is 1 to 10,000 (default 1,000). Pass next_cursor back as cursor until has_more is false. Before the batch has ended this answers 409, and after the results window it answers 410.

End to end in code

The OpenAI SDK’s client.batches uploads a JSONL file first and will not work against /v3/batches. Use plain HTTP:

List and delete batches

DELETE works only once the batch has ended; before that it answers 409. It deletes the batch and its stored results right away, instead of waiting for the results window.

Coming from OpenRouter

The request body is the same. Change the base URL and the key:
The batch object and result lines have the fields OpenRouter returns. callback_url works too and sends the same four event types, but the webhook is signed with Standard Webhooks headers and your project’s Opper secret, so verify it with a Standard Webhooks library rather than OpenRouter’s signature check.

Find the models you can batch

In the catalog, batch is a service tier with its own endpoints and prices:
A model with no batch endpoint answers 404 on submit. Batch endpoints follow your compliance rules: if your rules exclude them, the batch is refused.

Batch in the EU

Four providers run batches in the EU. You need no cloud account, bucket or role for either: send the batch as usual and name an EU batch endpoint, or set an EU inference location rule so only EU endpoints are allowed.

On Google Vertex AI

Claude Haiku 5.5, Claude Sonnet 5.5, Claude Opus 5.5, Gemini 3.8 Flash and Gemini 3.5 Flash Lite batches run on Vertex AI at half the EU price. Gemini runs in Google’s EU multi-region and Claude in europe-west4 (Netherlands). The endpoints are vertexai-batch/<model>:
On Vertex AI, a Gemini body may not set labels (the batch answers 400), and serviceTier is dropped, since a batch has its own tier.

On AWS Bedrock

Claude Haiku 4.5 and Claude Opus 5 batches run on AWS Bedrock in Stockholm (eu-north-1), through Bedrock’s EU cross-region inference, at half the EU on-demand price.
A Bedrock batch needs at least 100 requests. A smaller batch goes to another batch endpoint of the model if one is allowed, and otherwise answers 400 (Bedrock batches need at least 100 requests; this batch has N).

On Sference

Sference runs batches on its own Batch API in the EU, with requests sent inline. The endpoints are sference-batch/<model>:
  • sference-batch/deepseek-v4-flash-0731
  • sference-batch/deepseek-v4.1-flash-1017
  • sference-batch/glm-5.3
  • sference-batch/glm-5.3-flash
  • sference-batch/kimi-k3
The window is 24 hours. Sference batch is not zero data retention: see Data handling.

On Scaleway

Scaleway runs batches on its Generative APIs batch, processed in Paris (France), at half the realtime price on most models. The endpoints are scaleway-batch/<model>:
  • scaleway-batch/deepseek-v4-flash-0731
  • scaleway-batch/gemma-4-26b-a4b-it
  • scaleway-batch/glm-5.2
  • scaleway-batch/gpt-oss-120b
  • scaleway-batch/llama-3.3-70b-instruct
  • scaleway-batch/mistral-medium-3.5-128b
  • scaleway-batch/mistral-small-3.2-24b-instruct-2506
  • scaleway-batch/qwen3-235b-a22b-instruct-2507
  • scaleway-batch/qwen3.5-397b-a17b
  • scaleway-batch/qwen3.6-35b-a3b
  • scaleway-batch/qwen3.8-27b
The window is 24 hours. Scaleway prices are in EUR, and the batch is billed in USD at the day’s ECB rate. List the EU batch endpoints:

Data handling

Batch is not zero data retention. Your requests and results have to sit at rest while the job runs, and every provider’s batch API is outside its zero data retention program.
  • At the provider. The provider holds the requests and results while the batch runs. Opper deletes the provider’s copy as soon as it has collected the results.
  • At Opper, input. Requests are staged until the provider accepts the job, then deleted.
  • On AWS Bedrock. The job’s input and output files sit in Opper’s S3 bucket in Stockholm. They are deleted once the results are collected, and any file left behind expires after 7 days.
  • On Google Vertex AI. The job’s input and output files sit in Opper’s Cloud Storage bucket in the EU, encrypted at rest. They are deleted, with the job, once the results are collected, and any file left behind expires after 7 days. As on Vertex AI’s synchronous endpoints, Google may log a prompt its abuse classifiers flag for up to 90 days.
  • On Google’s Gemini API. Processing and storage are global, not EU. Google logs prompts and responses for 55 days for abuse monitoring. Opper deletes the input file and the batch once the results are collected; otherwise Google deletes them after 48 hours.
  • On Mistral. Batch runs only on Mistral’s global API, with no EU processing commitment, and is outside Mistral’s zero data retention. Mistral may keep requests and results for 30 days for abuse monitoring. Opper deletes the files and the job once the results are collected.
  • On Sference. Sference keeps batch results. The zero data retention of Opper’s synchronous Sference endpoints does not apply to batch.
  • On Scaleway. The input and results are stored encrypted in Opper’s Scaleway Object Storage bucket in Paris until Opper collects the results, then deleted at Opper and at Scaleway. Scaleway keeps the batch input for up to 24 hours while it processes the job.
  • At Opper, results. Kept for your project’s retention period after the batch ends, at most 30 days (24 hours if your project’s retention is 0), or until you DELETE the batch. results_expire_at on the batch object says when. After that, results answer 410 and the batch object keeps only its counts and cost (results_deleted: true).

Limits

Billing

A batch is billed once, when it finishes, at the batch endpoint’s price: about half the synchronous price. Each successful line is priced from its usage exactly like the synchronous call it stands for, and the batch is charged as one usage event. usage.cost on the batch object is the total. An expired or cancelled batch is billed only for the lines that finished. A batch that never started, or failed, is not charged. Because a batch is charged only when it finishes, it is accepted only if its worst case fits in what your organization can still spend: each request at its max_tokens (or the model’s maximum output when it sets none), plus the worst case of your batches still running, against your balance and any organization, project or member budget. Otherwise the batch answers 402 with estimated_cost_usd and available_usd; set max_tokens on the requests, top up, or split the batch. Batches need a paid plan. Tools the provider runs and charges per use (web search and fetch, file search, code execution, image generation, remote MCP) are not supported in a batch: a request that turns one on answers 400. Function tools and client tools work. A batch cannot refer to state stored at the provider: previous_response_id, conversation, a stored prompt, container, store: true, or a file_id in the input answers 400. Send the content inline (base64 or a data: URL) instead.
Last modified on November 11, 2026