> ## Documentation Index
> Fetch the complete documentation index at: https://docs.opper.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Batch API

> Send up to 50,000 requests in one job and get the results within 24 hours, at about half the synchronous price.

A batch is one job holding many requests to one model. Opper runs it on the provider's batch API, which costs about half the synchronous price and returns the results within 24 hours. You send the requests inline in one JSON body, poll the batch, and read the results when it ends.

> [!NOTE]
>
> Batch currently runs on OpenAI and Anthropic models, in the US. It is not available in the EU yet.

## Create a batch

```bash theme={null}
curl https://api.opper.ai/v3/batches \
  -H "Authorization: Bearer $OPPER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/v1/chat/completions",
    "model": "gpt-5.4-mini",
    "completion_window": "24h",
    "metadata": {"job": "ticket-triage"},
    "requests": [
      {"custom_id": "ticket-1", "body": {"messages": [{"role": "user", "content": "Classify: my invoice is wrong"}]}},
      {"custom_id": "ticket-2", "body": {"messages": [{"role": "user", "content": "Classify: the app crashes on login"}]}}
    ]
  }'
```

| Field | |
| - | - |
| `endpoint` | The format of every `body`: `/v1/chat/completions`, `/v1/responses` or `/v1/embeddings` for OpenAI models, `/v1/messages` for Claude models. |
| `model` | One model per batch. A model name, or a batch endpoint id to pin one. |
| `provider` | Optional. `{"only": [...]}` limits the batch to the batch endpoints or providers you list. |
| `completion_window` | `"24h"`, the only window. |
| `metadata` | Optional string key-value pairs, returned on the batch object. |
| `requests` | `[{custom_id, body}]`. `custom_id` is 1 to 64 letters, digits, `-` or `_`, and is how you match results to requests. |

Each `body` is the normal request for that endpoint and goes to the provider as it is. Opper sets `model` in each body for you, so you can leave it out. For a Claude batch, each body is an Anthropic Messages request:

```json theme={null}
{
  "endpoint": "/v1/messages",
  "model": "claude-sonnet-5-5",
  "completion_window": "24h",
  "requests": [
    {"custom_id": "doc-1", "body": {"max_tokens": 1024, "messages": [{"role": "user", "content": "Summarize: ..."}]}}
  ]
}
```

A body in another format than the model's batch endpoint takes (a chat completions body for a Claude model, for example) answers `400` naming the endpoints the model takes. `stream: true` and `callback_url` also answer `400`.

The response is `202` with the batch object, in status `validating`. Every `custom_id` and body is checked before anything is accepted: if one line is malformed, the whole batch is refused. A body the provider rejects later comes back as that line's error in the results.

## Poll the batch

```bash theme={null}
curl https://api.opper.ai/v3/batches/$BATCH_ID \
  -H "Authorization: Bearer $OPPER_API_KEY"
```

The batch object carries the fields of OpenAI's batch object plus OpenRouter's, so a client written for either can read it:

```json theme={null}
{
  "id": "batch_...",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "model": "gpt-5.4-mini",
  "provider": "openai-batch/gpt-5.4-mini",
  "completion_window": "24h",
  "status": "completed",
  "request_counts": {"total": 2, "completed": 2, "failed": 0},
  "usage": {"prompt_tokens": 40, "completion_tokens": 12, "total_tokens": 52, "cost": 0.00002},
  "metadata": {"job": "ticket-triage"},
  "created_at": 1793000000,
  "completed_at": 1793000420,
  "expires_at": 1793086400,
  "results_expire_at": 1793605220,
  "results": [ ... ]
}
```

| Status | Meaning |
| - | - |
| `validating` | Accepted, not yet handed to the provider. |
| `in_progress` | The provider is running it. |
| `finalizing` | The provider is done; Opper is collecting and pricing the results. |
| `completed` | Results are ready. |
| `expired` | The 24 hour window ended first. Results hold the lines that finished. |
| `cancelled` | The provider cancelled the job. Results hold the lines that finished. |
| `failed` | The batch could not run. `error` says why. There are no results. |

There is no cancel endpoint. Most batches finish in minutes; poll every 30 to 60 seconds.

## Read the results

Once the batch has ended (`completed`, `expired` or `cancelled`), `GET /v3/batches/{id}` includes `results` inline. Each line is OpenAI's batch result line, with the provider's response body as it is:

```json theme={null}
{
  "id": "...",
  "custom_id": "ticket-1",
  "response": {"status_code": 200, "request_id": "...", "body": {"choices": [...], "usage": {...}}},
  "error": null
}
```

Results are not in request order: match them by `custom_id`. For a large batch, page through `GET /v3/batches/{id}/results` instead of reading them all in one response:

```bash theme={null}
curl "https://api.opper.ai/v3/batches/$BATCH_ID/results?limit=1000" \
  -H "Authorization: Bearer $OPPER_API_KEY"
# {"object": "list", "data": [...], "has_more": true, "next_cursor": "1000"}

curl "https://api.opper.ai/v3/batches/$BATCH_ID/results?limit=1000&cursor=1000" \
  -H "Authorization: Bearer $OPPER_API_KEY"
```

`limit` is 1 to 10,000 (default 1,000). Pass `next_cursor` back as `cursor` until `has_more` is `false`. Before the batch has ended this answers `409`, and after the results window it answers `410`.

## End to end in code

The OpenAI SDK's `client.batches` uploads a JSONL file first and will not work against `/v3/batches`. Use plain HTTP:

```python Python (requests) theme={null}
import os, time, requests

BASE = "https://api.opper.ai/v3"
HEADERS = {"Authorization": f"Bearer {os.environ['OPPER_API_KEY']}"}

texts = ["my invoice is wrong", "the app crashes on login"]
batch = requests.post(f"{BASE}/batches", headers=HEADERS, json={
    "endpoint": "/v1/chat/completions",
    "model": "gpt-5.4-mini",
    "completion_window": "24h",
    "requests": [
        {"custom_id": f"ticket-{i}", "body": {"messages": [{"role": "user", "content": f"Classify: {t}"}]}}
        for i, t in enumerate(texts)
    ],
}).json()

while batch["status"] not in ("completed", "expired", "cancelled", "failed"):
    time.sleep(30)
    batch = requests.get(f"{BASE}/batches/{batch['id']}", headers=HEADERS).json()

for line in batch.get("results") or []:
    if line["error"]:
        print(line["custom_id"], "error:", line["error"]["message"])
    else:
        print(line["custom_id"], line["response"]["body"]["choices"][0]["message"]["content"])
```

```typescript TypeScript (fetch) theme={null}
const BASE = "https://api.opper.ai/v3";
const headers = {
  Authorization: `Bearer ${process.env.OPPER_API_KEY}`,
  "Content-Type": "application/json",
};

const texts = ["my invoice is wrong", "the app crashes on login"];
let batch = await (await fetch(`${BASE}/batches`, {
  method: "POST",
  headers,
  body: JSON.stringify({
    endpoint: "/v1/chat/completions",
    model: "gpt-5.4-mini",
    completion_window: "24h",
    requests: texts.map((t, i) => ({
      custom_id: `ticket-${i}`,
      body: { messages: [{ role: "user", content: `Classify: ${t}` }] },
    })),
  }),
})).json();

const ended = ["completed", "expired", "cancelled", "failed"];
while (!ended.includes(batch.status)) {
  await new Promise((r) => setTimeout(r, 30_000));
  batch = await (await fetch(`${BASE}/batches/${batch.id}`, { headers })).json();
}

for (const line of batch.results ?? []) {
  console.log(line.custom_id, line.error ?? line.response.body.choices[0].message.content);
}
```

## List and delete batches

```bash theme={null}
# The project's batches, newest first. Page with ?after=<last_id>&limit= (1 to 100, default 20)
curl https://api.opper.ai/v3/batches -H "Authorization: Bearer $OPPER_API_KEY"

# Delete a batch that has ended, and its results
curl -X DELETE https://api.opper.ai/v3/batches/$BATCH_ID -H "Authorization: Bearer $OPPER_API_KEY"
```

`DELETE` works only once the batch has ended; before that it answers `409`. It deletes the batch and its stored results right away, instead of waiting for the results window.

## Coming from OpenRouter

The request body is the same. Change the base URL and the key:

```diff theme={null}
- https://openrouter.ai/api/v1/batches
+ https://api.opper.ai/v3/batches
- Authorization: Bearer $OPENROUTER_API_KEY
+ Authorization: Bearer $OPPER_API_KEY
```

The batch object and result lines have the fields OpenRouter returns. `callback_url` is not supported yet and answers `400`; poll instead.

## Find the models you can batch

In the catalog, batch is a [service tier](https://docs.opper.ai/build/gateway/service-tiers.md) with its own endpoints and prices:

```bash theme={null}
curl "https://api.opper.ai/v3/models?service_tier=batch&limit=0"
```

A model with no batch endpoint answers `404` on submit. Batch endpoints follow your [compliance rules](https://docs.opper.ai/control-plane/rules/overview.md): if your rules exclude them, the batch is refused.

## Data handling

> [!WARNING]
>
> Batch is not zero data retention. Your requests and results have to sit at rest while the job runs, and OpenAI's and Anthropic's batch APIs are outside their zero data retention programs.

* **At the provider.** The provider holds the requests and results while the batch runs. Opper deletes the provider's copy as soon as it has collected the results.
* **At Opper, input.** Requests are staged until the provider accepts the job, then deleted.
* **At Opper, results.** Kept for your project's retention period after the batch ends, at most 30 days (24 hours if your project's retention is 0), or until you `DELETE` the batch. `results_expire_at` on the batch object says when. After that, results answer `410` and the batch object keeps only its counts and cost (`results_deleted: true`).

## Limits

| | |
| - | - |
| Requests per batch | 50,000. More answers `413`: split the batch. |
| Request body size | 100 MiB. More answers `413`. |
| Models per batch | One. |
| Completion window | 24 hours. |
| Results kept | Your project's retention period after the batch ends, at most 30 days (24 hours on retention 0). |
| Regions | US. Not available in the EU yet. |

## Billing

A batch is billed once, when it finishes, at the batch endpoint's price: about half the synchronous price. Each successful line is priced from its usage exactly like the synchronous call it stands for, and the batch is charged as one usage event. `usage.cost` on the batch object is the total.

An `expired` or `cancelled` batch is billed only for the lines that finished. A batch that never started, or `failed`, is not charged. You need a positive balance to submit a batch, as for any call.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.