Batch API - Erebine

The workflow is:

Upload, Upload a JSONL file containing your requests via the Files API.
Create, Create a batch referencing the uploaded file.
Poll, Check batch status until it reaches a terminal state.
Download, Download the output file containing results.

Batch requests share the same API rate limit quotas as synchronous traffic and are billed at the base per-token rate for your endpoint's tier. Unlike some providers, no batch-tier pricing discount is applied. Batch input and output files are stored using the platform's two-tier storage architecture. For details on storage tiers, encryption, retention, and billing, see Storage.

Input file size limit: Batch input files must not exceed 100 MB. Larger files may upload successfully through the Files API (which permits up to 500 MB) but will fail batch validation with status: "failed".

Supported Endpoints

Batch processing supports the following target endpoints:

Endpoint	Description
`/v1/chat/completions`	Batch chat completion requests
`/v1/embeddings`	Batch embedding generation
`/v1/responses`	Batch Responses API processing

All lines within a single batch must target the same endpoint. The target endpoint is specified both in the batch creation request and in each JSONL line's url field.

JSONL Format

Each line in the input file must be a valid JSON object with the following fields:

Field	Type	Description
custom_idrequired	string	Unique correlation ID. Must be unique within the file.
methodrequired	string	Must be `"POST"`.
urlrequired	string	Target endpoint path (e.g., `/v1/chat/completions`).
bodyrequired	object	Request body matching the target endpoint format.

Chat Completions Example

JSONL

{
  "custom_id": "req-1",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "llama-3.1-70b",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 100
  }
}

{
  "custom_id": "req-2",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "llama-3.1-70b",
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "max_tokens": 100
  }
}
                

Embeddings example

JSONL

{
  "custom_id": "emb-1",
  "method": "POST",
  "url": "/v1/embeddings",
  "body": {
    "model": "bge-large-en-v1.5",
    "input": "First document text"
  }
}

{
  "custom_id": "emb-2",
  "method": "POST",
  "url": "/v1/embeddings",
  "body": {
    "model": "bge-large-en-v1.5",
    "input": "Second document text"
  }
}
                    

Responses API example

JSONL

{
  "custom_id": "resp-1",
  "method": "POST",
  "url": "/v1/responses",
  "body": {
    "model": "llama-3.1-8b",
    "input": "Summarize this document."
  }
}

{
  "custom_id": "resp-2",
  "method": "POST",
  "url": "/v1/responses",
  "body": {
    "model": "llama-3.1-8b",
    "input": "Translate to French: Hello world"
  }
}
                    

Note: Batch Responses API requests do not support background: true or stream: true. The store parameter defaults to false for batch responses.

Line-level errors: Malformed JSONL lines and duplicate custom_id values do not cause POST /v1/batches to return 400. The create call succeeds, then the batch advances through validating and terminates in failed with the per-line problems surfaced on the batch's top-level errors field (or, for partial failures during processing, in the error_file_id JSONL).

Create Batch

POST /:project_id/:endpoint_slug/v1/batches

Batch creation is endpoint-scoped: the path names the endpoint slug the batch runs against. An OpenAI SDK configured with base_url = https://api.erebine.ai/proj_ABC123/my-endpoint/v1 posts to this route with no code changes. Listing, retrieving, and cancelling batches are not endpoint-scoped: they reach the batches of the workspace the request resolves to (see Workspace Scope).

Parameter	Type	Description
input_file_idrequired	string	ID of the uploaded JSONL file.
endpointrequired	string	Target endpoint: `/v1/chat/completions`, `/v1/embeddings`, or `/v1/responses`.
completion_windowrequired	string	Must be the string `"24h"`, not the integer `24`.
metadataoptional	object	Custom key-value pairs. The serialized JSON object must not exceed 16 KB.

curl

                    curl -X POST https://api.erebine.ai/proj_ABC123/my-endpoint/v1/batches \
  -H "Authorization: Bearer ere_myproject_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123def456",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h",
    "metadata": {"description": "nightly evaluation"}
  }'
                

Response

                        {
  "id": "batch_abc123def456",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "input_file_id": "file-abc123def456",
  "output_file_id": null,
  "error_file_id": null,
  "status": "validating",
  "completion_window": "24h",
  "created_at": 1709000000,
  "in_progress_at": null,
  "expires_at": 1709086400,
  "finalizing_at": null,
  "completed_at": null,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": null,
  "cancelled_at": null,
  "metadata": {"description": "nightly evaluation"},
  "request_counts": {
    "total": 0,
    "completed": 0,
    "failed": 0
  },
  "errors": null,
  "usage": null
}
                    

Response Fields

Field	Type	Description
id	string	External batch identifier (e.g., `batch_abc123`).
status	string	Current batch status. See Status Lifecycle.
input_file_id	string	External ID of the input JSONL file.
output_file_id	string \| null	External ID of the output JSONL file. Set once the output upload completes, typically while the batch is in the `finalizing` state and always by the time it reaches `completed`.
error_file_id	string \| null	External ID of the error JSONL file. Set if any requests failed.
in_progress_at	integer \| null	Unix timestamp when processing started.
expires_at	integer \| null	Unix timestamp when the 24h completion window expires.
finalizing_at	integer \| null	Unix timestamp when the batch entered the finalizing state.
completed_at	integer \| null	Unix timestamp when the batch completed successfully.
failed_at	integer \| null	Unix timestamp when the batch failed.
expired_at	integer \| null	Unix timestamp when the batch actually expired (distinct from `expires_at`).
cancelling_at	integer \| null	Unix timestamp when cancellation was requested.
cancelled_at	integer \| null	Unix timestamp when cancellation completed.
request_counts	object	Processing counts: `total`, `completed`, `failed`.
errors	object \| null	Top-level error information if the batch itself failed. Contains `object: "list"` and `data` array of error records.
usage	object \| null	Aggregate token usage when requests have completed: `total_tokens`, `prompt_tokens`, `completion_tokens`.

List Batches

GET /:project_id/v1/batches

curl

                    curl https://api.erebine.ai/proj_ABC123/v1/batches?limit=20 \
  -H "Authorization: Bearer ere_myproject_your_api_key"
                

Get Batch

GET /:project_id/v1/batches/{batch_id}

curl

                    curl https://api.erebine.ai/proj_ABC123/v1/batches/batch_abc123def456 \
  -H "Authorization: Bearer ere_myproject_your_api_key"
                

Cancel Batch

POST /:project_id/v1/batches/{batch_id}/cancel

curl

                    curl -X POST https://api.erebine.ai/proj_ABC123/v1/batches/batch_abc123def456/cancel \
  -H "Authorization: Bearer ere_myproject_your_api_key"
                

Cancellation stops dispatching new requests. In-flight requests complete normally. The batch transitions to cancelled once all in-flight requests finish.

Workspace Scope

A batch belongs to the workspace its create request resolved to: the X-Workspace-Id header, else the only workspace of an API key confined to one, else the project's default workspace. Its input file must belong to the same workspace, and its output and error files do. Listing, retrieving and cancelling resolve the workspace the same way and reach only that workspace's batches; a batch in another workspace answers 404, exactly as a missing one does.

Every request in a batch runs as the key that created the batch, in the batch's workspace: with that key's workspace list, scopes and user, billed and rate-limited as the key's own requests are. A headers member on a line is ignored, and an input_image part that names a file by file_id reaches only the files the key reaches in the batch's workspace (see Files). Each request is checked when it runs, not only when the batch is created: once the key is revoked or expires, its user is suspended, leaves the project or can no longer read the workspace, or its workspace list no longer includes the workspace, the remaining requests fail with the error the key's own request gets (for example key_inactive or workspace_not_allowed) and appear in the error file. A request refused 429 by the key's rate limit or quota waits and runs later, within the completion window.

An API key confined to some of the project's workspaces creates and reaches batches only in those. Naming another returns 403 Forbidden (workspace_not_allowed), and a key confined to several workspaces must name one. A key that belongs to a user creates and reaches batches only in workspaces that user can read. See Authentication & Security.
A workspace in zero-data-retention mode cannot run batches: the create returns 400 Bad Request (retention_not_permitted). If the batch's workspace enters the mode before the batch finishes, the remaining requests fail with the same code, and a batch that finishes under it keeps no results and ends failed.
Batches created before batches recorded their workspace belong to none. Only a key with no user behind it and no workspace list reaches them, and their pending requests fail with batch_context_missing.

Output Format

When a batch completes, the output_file_id references a JSONL file containing results. Each line corresponds to a request:

Successful Request

JSONL

{
  "id": "batch_req_abc123",
  "custom_id": "req-1",
  "response": {
    "status_code": 200,
    "request_id": "req-xyz",
    "body": {
      "id": "chatcmpl-abc",
      "object": "chat.completion",
      "choices": [...],
      "usage": {...}
    }
  },
  "error": null
}
                

Failed Request

Failed requests appear in the error_file_id JSONL file:

JSONL

{
  "id": "batch_req_def456",
  "custom_id": "req-2",
  "response": null,
  "error": {
    "code": "model_not_found",
    "message": "Requested model is not available on this endpoint",
    "param": "model",
    "line": 42
  }
}
                

Field	Type	Description
code	string	Canonical machine-readable code. See Error Handling.
message	string	Human-readable explanation of the failure.
paramoptional	string	Offending request field when applicable.
lineoptional	integer	1-based line number in the original input JSONL.

Download the output file using the Files API (see Download File Content):

curl

                    curl https://api.erebine.ai/proj_ABC123/v1/files/file-output789/content \
  -H "Authorization: Bearer ere_myproject_your_api_key" \
  -o results.jsonl
                

Status Lifecycle

Status	Description
`validating`	Input file is being parsed and validated.
`in_progress`	Requests are being processed.
`finalizing`	Building output and error files.
`completed`	All requests processed. Output files available.
`failed`	Unrecoverable error during processing.
`expired`	24-hour deadline exceeded.
`cancelling`	Cancel requested, draining in-flight requests.
`cancelled`	Cancellation complete.

CLI Commands

The erectl CLI provides batch management commands. The full flag surface lives in erectl batches.

Full batch workflow

                    # Upload input file
erectl files --upload requests.jsonl --purpose batch

# Submit a batch against an endpoint slug
erectl batches --endpoint my-endpoint --input requests.jsonl

# Check batch status
erectl batches --endpoint my-endpoint --get batch_abc123def456

# List all batches for an endpoint
erectl batches --endpoint my-endpoint --list

# Cancel a batch
erectl batches --endpoint my-endpoint --cancel batch_abc123def456

# Download output when complete
erectl files --download file-output789 --output results.jsonl
                

CLI endpoint coverage: The erectl batches command currently submits batches against /v1/chat/completions only. The HTTP API supports /v1/embeddings and /v1/responses as batch targets as well (see Supported Endpoints); to submit those today, call the REST API directly using the curl or client examples on this page. Confirm the exact flag spelling for your build with erectl batches --help and erectl files --help.

Client Examples

Python (requests)

Python

                    import requests

headers = {
    "Authorization": "Bearer ere_myproject_your_api_key",
    "Content-Type": "application/json"
}
base = "https://api.erebine.ai/proj_ABC123/v1"

# Step 1: Upload a JSONL input file
with open("requests.jsonl", "rb") as f:
    upload = requests.post(
        f"{base}/files",
        headers={"Authorization": headers["Authorization"]},
        files={"file": ("requests.jsonl", f)},
        data={"purpose": "batch"}
    ).json()
print(f"Uploaded file: {upload['id']}")

# Step 2: Create a batch
batch = requests.post(f"{base}/batches", headers=headers, json={
    "input_file_id": upload["id"],
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h",
    "metadata": {"description": "nightly evaluation"}
}).json()
print(f"Batch created: {batch['id']} (status: {batch['status']})")

# Step 3: Poll for completion
import time
while True:
    status = requests.get(
        f"{base}/batches/{batch['id']}",
        headers=headers
    ).json()
    print(f"Status: {status['status']} ({status['request_counts']})")
    if status["status"] in ("completed", "failed", "expired", "cancelled"):
        break
    time.sleep(10)

# Step 4: Download results
if status.get("output_file_id"):
    output = requests.get(
        f"{base}/files/{status['output_file_id']}/content",
        headers={"Authorization": headers["Authorization"]}
    )
    with open("results.jsonl", "wb") as f:
        f.write(output.content)
    print("Results saved to results.jsonl")

# List batches
batches = requests.get(f"{base}/batches?limit=20", headers=headers).json()
for b in batches["data"]:
    print(f"{b['id']} - {b['status']}")

# Cancel a batch
requests.post(f"{base}/batches/{batch['id']}/cancel", headers=headers)
                

Node.js (fetch)

JavaScript

                    import { readFile, writeFile } from "node:fs/promises";

const base = "https://api.erebine.ai/proj_ABC123/v1";
const headers = {
    "Authorization": "Bearer ere_myproject_your_api_key",
    "Content-Type": "application/json"
};

// Step 1: Upload a JSONL input file
const fileData = await readFile("requests.jsonl");
const formData = new FormData();
formData.append("file", new Blob([fileData]), "requests.jsonl");
formData.append("purpose", "batch");

const uploadRes = await fetch(`${base}/files`, {
    method: "POST",
    headers: { "Authorization": headers.Authorization },
    body: formData
});
const upload = await uploadRes.json();
console.log(`Uploaded file: ${upload.id}`);

// Step 2: Create a batch
const batchRes = await fetch(`${base}/batches`, {
    method: "POST",
    headers,
    body: JSON.stringify({
        input_file_id: upload.id,
        endpoint: "/v1/chat/completions",
        completion_window: "24h",
        metadata: { description: "nightly evaluation" }
    })
});
const batch = await batchRes.json();
console.log(`Batch created: ${batch.id} (${batch.status})`);

// Step 3: Poll for completion
let status;
while (true) {
    const res = await fetch(`${base}/batches/${batch.id}`, { headers });
    status = await res.json();
    console.log(`Status: ${status.status}`);
    if (["completed", "failed", "expired", "cancelled"].includes(status.status)) break;
    await new Promise(r => setTimeout(r, 10000));
}

// Step 4: Download results
if (status.output_file_id) {
    const outputRes = await fetch(
        `${base}/files/${status.output_file_id}/content`,
        { headers: { "Authorization": headers.Authorization } }
    );
    const outputData = await outputRes.arrayBuffer();
    await writeFile("results.jsonl", Buffer.from(outputData));
    console.log("Results saved to results.jsonl");
}
                

Error Handling

HTTP Status	Error Code	Description
400	`invalid_request`	Malformed request or invalid parameters.
400	`validation_error`	JSONL validation failed (malformed lines, duplicate custom_ids, etc.).
400	`retention_not_permitted`	The batch's workspace is in zero-data-retention mode. See Workspace Scope.
401	`authentication_error`	Invalid or missing API key.
403	`workspace_not_allowed`	The request names a workspace the API key may not use (outside its workspace list, or one its user cannot read), or a key confined to several workspaces names none. See Workspace Scope.
403	`workspace_required`	The project has no default workspace and the request, made with a key that belongs to a user, names none.
404	`not_found`	Batch or input file not found, or not in the request's workspace.
429	`rate_limit_exceeded`	Too many requests. Check the `Retry-After` header.