Batch API

A batch is one job holding many requests to one model. Opper runs it on the provider’s batch API, which costs about half the synchronous price and returns the results within 24 hours. You send the requests inline in one JSON body, poll the batch, and read the results when it ends.

Batch currently runs on OpenAI and Anthropic models, in the US. It is not available in the EU yet.

Create a batch

curl https://api.opper.ai/v3/batches \
  -H "Authorization: Bearer $OPPER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/v1/chat/completions",
    "model": "gpt-5.4-mini",
    "completion_window": "24h",
    "metadata": {"job": "ticket-triage"},
    "requests": [
      {"custom_id": "ticket-1", "body": {"messages": [{"role": "user", "content": "Classify: my invoice is wrong"}]}},
      {"custom_id": "ticket-2", "body": {"messages": [{"role": "user", "content": "Classify: the app crashes on login"}]}}
    ]
  }'

Field
`endpoint`	The format of every `body`: `/v1/chat/completions`, `/v1/responses` or `/v1/embeddings` for OpenAI models, `/v1/messages` for Claude models.
`model`	One model per batch. A model name, or a batch endpoint id to pin one.
`provider`	Optional. `{"only": [...]}` limits the batch to the batch endpoints or providers you list.
`completion_window`	`"24h"`, the only window.
`metadata`	Optional string key-value pairs, returned on the batch object.
`requests`	`[{custom_id, body}]`. `custom_id` is 1 to 64 letters, digits, `-` or `_`, and is how you match results to requests.

Each body is the normal request for that endpoint and goes to the provider as it is. Opper sets model in each body for you, so you can leave it out. For a Claude batch, each body is an Anthropic Messages request:

{
  "endpoint": "/v1/messages",
  "model": "claude-sonnet-5-5",
  "completion_window": "24h",
  "requests": [
    {"custom_id": "doc-1", "body": {"max_tokens": 1024, "messages": [{"role": "user", "content": "Summarize: ..."}]}}
  ]
}

A body in another format than the model’s batch endpoint takes (a chat completions body for a Claude model, for example) answers 400 naming the endpoints the model takes. stream: true and callback_url also answer 400. The response is 202 with the batch object, in status validating. Every custom_id and body is checked before anything is accepted: if one line is malformed, the whole batch is refused. A body the provider rejects later comes back as that line’s error in the results.

Poll the batch

curl https://api.opper.ai/v3/batches/$BATCH_ID \
  -H "Authorization: Bearer $OPPER_API_KEY"

The batch object carries the fields of OpenAI’s batch object plus OpenRouter’s, so a client written for either can read it:

{
  "id": "batch_...",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "model": "gpt-5.4-mini",
  "provider": "openai-batch/gpt-5.4-mini",
  "completion_window": "24h",
  "status": "completed",
  "request_counts": {"total": 2, "completed": 2, "failed": 0},
  "usage": {"prompt_tokens": 40, "completion_tokens": 12, "total_tokens": 52, "cost": 0.00002},
  "metadata": {"job": "ticket-triage"},
  "created_at": 1793000000,
  "completed_at": 1793000420,
  "expires_at": 1793086400,
  "results_expire_at": 1793605220,
  "results": [ ... ]
}

Status	Meaning
`validating`	Accepted, not yet handed to the provider.
`in_progress`	The provider is running it.
`finalizing`	The provider is done; Opper is collecting and pricing the results.
`completed`	Results are ready.
`expired`	The 24 hour window ended first. Results hold the lines that finished.
`cancelled`	The provider cancelled the job. Results hold the lines that finished.
`failed`	The batch could not run. `error` says why. There are no results.

There is no cancel endpoint. Most batches finish in minutes; poll every 30 to 60 seconds.

Read the results

Once the batch has ended (completed, expired or cancelled), GET /v3/batches/{id} includes results inline. Each line is OpenAI’s batch result line, with the provider’s response body as it is:

{
  "id": "...",
  "custom_id": "ticket-1",
  "response": {"status_code": 200, "request_id": "...", "body": {"choices": [...], "usage": {...}}},
  "error": null
}

Results are not in request order: match them by custom_id. For a large batch, page through GET /v3/batches/{id}/results instead of reading them all in one response:

curl "https://api.opper.ai/v3/batches/$BATCH_ID/results?limit=1000" \
  -H "Authorization: Bearer $OPPER_API_KEY"
# {"object": "list", "data": [...], "has_more": true, "next_cursor": "1000"}

curl "https://api.opper.ai/v3/batches/$BATCH_ID/results?limit=1000&cursor=1000" \
  -H "Authorization: Bearer $OPPER_API_KEY"

limit is 1 to 10,000 (default 1,000). Pass next_cursor back as cursor until has_more is false. Before the batch has ended this answers 409, and after the results window it answers 410.

End to end in code

The OpenAI SDK’s client.batches uploads a JSONL file first and will not work against /v3/batches. Use plain HTTP:

import os, time, requests

BASE = "https://api.opper.ai/v3"
HEADERS = {"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}

texts = ["my invoice is wrong", "the app crashes on login"]
batch = requests.post(f"{BASE}/batches", headers=HEADERS, json={
    "endpoint": "/v1/chat/completions",
    "model": "gpt-5.4-mini",
    "completion_window": "24h",
    "requests": [
        {"custom_id": f"ticket-{i}", "body": {"messages": [{"role": "user", "content": f"Classify: {t}"}]}}
        for i, t in enumerate(texts)
    ],
}).json()

while batch["status"] not in ("completed", "expired", "cancelled", "failed"):
    time.sleep(30)
    batch = requests.get(f"{BASE}/batches/{batch['id']}", headers=HEADERS).json()

for line in batch.get("results") or []:
    if line["error"]:
        print(line["custom_id"], "error:", line["error"]["message"])
    else:
        print(line["custom_id"], line["response"]["body"]["choices"][0]["message"]["content"])

const BASE = "https://api.opper.ai/v3";
const headers = {
  Authorization: `Bearer ${process.env.OPPER_API_KEY}`,
  "Content-Type": "application/json",
};

const texts = ["my invoice is wrong", "the app crashes on login"];
let batch = await (await fetch(`${BASE}/batches`, {
  method: "POST",
  headers,
  body: JSON.stringify({
    endpoint: "/v1/chat/completions",
    model: "gpt-5.4-mini",
    completion_window: "24h",
    requests: texts.map((t, i) => ({
      custom_id: `ticket-${i}`,
      body: { messages: [{ role: "user", content: `Classify: ${t}` }] },
    })),
  }),
})).json();

const ended = ["completed", "expired", "cancelled", "failed"];
while (!ended.includes(batch.status)) {
  await new Promise((r) => setTimeout(r, 30_000));
  batch = await (await fetch(`${BASE}/batches/${batch.id}`, { headers })).json();
}

for (const line of batch.results ?? []) {
  console.log(line.custom_id, line.error ?? line.response.body.choices[0].message.content);
}

List and delete batches

# The project's batches, newest first. Page with ?after=<last_id>&limit= (1 to 100, default 20)
curl https://api.opper.ai/v3/batches -H "Authorization: Bearer $OPPER_API_KEY"

# Delete a batch that has ended, and its results
curl -X DELETE https://api.opper.ai/v3/batches/$BATCH_ID -H "Authorization: Bearer $OPPER_API_KEY"

DELETE works only once the batch has ended; before that it answers 409. It deletes the batch and its stored results right away, instead of waiting for the results window.

Coming from OpenRouter

The request body is the same. Change the base URL and the key:

- https://openrouter.ai/api/v1/batches
+ https://api.opper.ai/v3/batches
- Authorization: Bearer $OPENROUTER_API_KEY
+ Authorization: Bearer $OPPER_API_KEY

The batch object and result lines have the fields OpenRouter returns. callback_url is not supported yet and answers 400; poll instead.

Find the models you can batch

In the catalog, batch is a service tier with its own endpoints and prices:

curl "https://api.opper.ai/v3/models?service_tier=batch&limit=0"

A model with no batch endpoint answers 404 on submit. Batch endpoints follow your compliance rules: if your rules exclude them, the batch is refused.

Data handling

Batch is not zero data retention. Your requests and results have to sit at rest while the job runs, and OpenAI’s and Anthropic’s batch APIs are outside their zero data retention programs.

At the provider. The provider holds the requests and results while the batch runs. Opper deletes the provider’s copy as soon as it has collected the results.
At Opper, input. Requests are staged until the provider accepts the job, then deleted.
At Opper, results. Kept for your project’s retention period after the batch ends, at most 30 days (24 hours if your project’s retention is 0), or until you DELETE the batch. results_expire_at on the batch object says when. After that, results answer 410 and the batch object keeps only its counts and cost (results_deleted: true).

Limits


Requests per batch	50,000. More answers `413`: split the batch.
Request body size	100 MiB. More answers `413`.
Models per batch	One.
Completion window	24 hours.
Results kept	Your project’s retention period after the batch ends, at most 30 days (24 hours on retention 0).
Regions	US. Not available in the EU yet.

Billing

A batch is billed once, when it finishes, at the batch endpoint’s price: about half the synchronous price. Each successful line is priced from its usage exactly like the synchronous call it stands for, and the batch is charged as one usage event. usage.cost on the batch object is the total. An expired or cancelled batch is billed only for the lines that finished. A batch that never started, or failed, is not charged. You need a positive balance to submit a batch, as for any call.

Get started

Platform

Build

Observe & Control

Apps

Tutorials

Tooling

Create a batch

Poll the batch

Read the results

End to end in code

List and delete batches

Coming from OpenRouter

Find the models you can batch

Data handling

Limits

Billing

​Create a batch

​Poll the batch

​Read the results

​End to end in code

​List and delete batches

​Coming from OpenRouter

​Find the models you can batch

​Data handling

​Limits

​Billing

Create a batch

Poll the batch

Read the results

End to end in code

List and delete batches

Coming from OpenRouter

Find the models you can batch

Data handling

Limits

Billing