docs
// Guides

SDK & Integrations

Point any OpenAI-compatible SDK at a Erebine base URL and ship. Same request shape, same response shape, passed through as sent, plus SLO headers, cached-token accounting, and a defined error envelope.

For practical how-to recipes covering streaming, rate limiting, error handling patterns, and log probabilities, see Usage Guides.

Model names are illustrative. Every code example on this page uses YOUR_MODEL_NAME as a placeholder. Erebine has no global catalog of pre-defined model ids; model names are defined per project on each endpoint. Discover the models available on your endpoint with GET /{project_id}/{endpoint_slug}/v1/models, then substitute the returned id wherever YOUR_MODEL_NAME appears below.
One client, one endpoint. The path segment before /v1 in the base URL is the endpoint slug. Unlike the OpenAI API, an instantiated SDK client is bound to a single endpoint, the "one client, switch models" pattern does not apply. Customers running multiple endpoints must instantiate one client per endpoint slug or rebuild the base URL per request.
Semantic-routed base URL. To route across every generative endpoint in a workspace instead of binding to one slug, swap the endpoint slug for WORKSPACE_ID/_semantic/router. The base URL becomes https://api.erebine.ai/proj_ABC123/WORKSPACE_ID/_semantic/router/v1, where WORKSPACE_ID is your ws_-prefixed workspace ID. The router picks the endpoint that best matches each request; see Semantic Router. It picks per request, so a multi-turn client should send a stable X-Conversation-Id to hold the thread on one model (Pin a Conversation), and read X-Erebine-Routed-Model to see what served each turn.

Migrate from OpenAI

Two changes. Base URL, API key. Same SDK, same response shape.

  1. Base URL: https://api.openai.com/v1 becomes https://api.erebine.ai/proj_ABC123/ENDPOINT_SLUG/v1.
  2. API key: sk-... becomes ere_{project_slug}_{random}. Create one in the dashboard or via POST /{project_id}/v1/management/api-keys.
Python (OpenAI SDK)
# Before from openai import OpenAI client = OpenAI(api_key="sk-...") response = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": "Hello!"}] ) # After from openai import OpenAI client = OpenAI( base_url="https://api.erebine.ai/proj_ABC123/my-endpoint/v1", api_key="ere_YOUR_PROJECT_SLUG_YOUR_API_KEY" ) response = client.chat.completions.create( model="YOUR_MODEL_NAME", messages=[{"role": "user", "content": "Hello!"}] )

Custom-domain endpoints substitute the host portion only; the /ENDPOINT_SLUG/v1 suffix remains. The endpoint slug is fixed at endpoint creation and surfaces in the dashboard URL list.

What Differs from OpenAI

A few behaviors worth knowing before you ship.

  • Faithful by default: an SDK request with no Erebine headers is served as sent. Your messages reach the model unchanged, max_tokens / tools / tool_choice are honored verbatim, and one request is one model turn. Erebine's agentic harness is opt-in via X-Erebine-Augment: on. See Passthrough and Augmentation.
  • Strict-template model families: a few model families -- Mistral and Gemma among them -- ship chat templates that reject message orderings OpenAI accepts: an assistant turn first, two user turns in a row, a system turn in the middle of the conversation, or a tool result placed immediately before a user turn. Rather than return a template error, Erebine normalizes those shapes with synthetic bridging turns so the request succeeds. It is a per-family repair, applied only to the families whose template needs it; every other model receives your messages unchanged.
  • service_tier in responses is the Erebine endpoint tier slug (e.g. gpu_nvidia_shared), not the OpenAI vocabulary (default / flex / scale). On requests it does not override endpoint tier but DOES influence routing priority scoring and billing within that tier.
  • stream_options.include_usage matches OpenAI: streaming usage is off unless you set stream_options.include_usage: true, in which case the final SSE chunk carries a usage object. Listed here because clients often expect a difference; there is none. Token counts are tracked internally for billing regardless.

Legacy /v1/completions and /v1/moderations are not implemented; chat completions, embeddings, audio, files, batch, and the responses API are. See API Reference for the full surface.

Passthrough and Augmentation

The OpenAI SDK expects the API to serve the request it sent. Erebine does. With no Erebine headers on the request:

  • Your messages array reaches the model unchanged, including your system prompt at index 0.
  • max_tokens, tools, tool_choice, and response_format are honored verbatim.
  • No tool call runs that the model did not emit.
  • One create() call is one model turn, and one bill.

Erebine's agentic harness -- parallel-tool-call nudging, corrective retries, research grounding and fallback search, context compaction, in-path token compression -- is opt-in. Send X-Erebine-Augment: on for the whole bundle, or a per-feature header for one piece of it. Every header is in the Header Reference below.

Python
from openai import OpenAI client = OpenAI( base_url="https://api.erebine.ai/proj_ABC123/my-endpoint/v1", api_key="ere_myproject_your_api_key", default_headers={"X-Erebine-Augment": "on"}, )

Errors Instead of Substitutions

Faithful mode has no repair path, so a request that cannot be served as specified returns a spec-shaped error rather than a different request's answer. Handle these the way you already handle OpenAI's:

  • 400 context_length_exceeded: the conversation plus the requested output does not fit the context window. Erebine does not drop your oldest turns and answer from a truncated conversation. Opt into X-Erebine-Augment-Compaction or X-Erebine-Augment-Max-Output-Tokens-Clamp if you want it to.
  • 400 unsupported_parameter: you sent tools, or tool-call history, to a model with no tool-call support. Erebine does not answer as though you sent none. Route to a tool-capable model.
  • 408 timeout: the worker exceeded the endpoint's request deadline.

Header Reference

Every header a client can send and every header Erebine sends back. This is the complete list; other pages describe these headers in their own context and link here.

Boolean Values

Every header that carries a flag takes the same vocabulary: on, 1, true, or yes to enable; off, 0, false, or no to disable. Case-insensitive. A value that is none of these is treated as if the header were absent, so a typo never flips a feature.

Precedence for the augmentation family: a per-feature header beats the X-Erebine-Augment bundle, which beats the default. The default for API traffic is off, so an SDK request that sends none of these headers is served exactly as sent.

Request Headers

Header Values Default What it does
X-Erebine-Augment boolean off The bundle. Turns every augmentation feature below on or off at once. Any per-feature header overrides it.
X-Erebine-Augment-Max-Tokens-Floor boolean bundle Raises max_tokens to a 4096 floor when the request declares tools. Off: your value stands, whatever it is.
X-Erebine-Augment-Compaction boolean bundle Drops your oldest turns and retries when the conversation overflows the context window or the worker times out. Off: the error is returned.
X-Erebine-Augment-Max-Output-Tokens-Clamp boolean bundle Lowers an over-budget max_tokens / max_completion_tokens / max_output_tokens to fit the remaining context. Off: 400 context_length_exceeded.
X-Erebine-Augment-Capability-Tool-Drop boolean bundle Serves a tool-bearing request on a model with no tool-call support by discarding the tools and answering without them. Off: 400.
X-Erebine-Augment-Corrective-Retries boolean bundle Repairs a malformed tool call, a schema-violating tool call, an empty completion, or a premature stop by writing a corrective message into your conversation and running another model turn. Each retry is an additional billed model turn; your max_tokens bounds the cumulative output across the whole request (not each turn), and the run stops with finish_reason: "length" once that budget is reached. Off: the turn is returned as the model produced it, billed once.
X-Erebine-Augment-Fanout-Nudge boolean bundle Injects a system directive ahead of your own system prompt that pushes the model to issue tool calls in parallel. The directive is emitted only when a turn advertises two or more server-owned tools (built-in research tools, or workspace / Responses server tools); a request whose tools are all your own client-side functions never triggers it, so the header is a safe no-op there. The header is always honored, never rejected. Off: index 0 of messages is yours.
X-Erebine-Augment-Fabricated-Tool-Calls boolean bundle Lets the router call tools the model never called -- the research grounding fallback, the workspace-briefing projector -- and splice their results into the conversation. Off: only the model's own tool calls run.
X-Erebine-Augment-Tool-Allowlist boolean bundle Intersects your tools array with the saved tool scope of the chat named by X-Chat-Id. Off: your tools are sent as declared.
X-Erebine-Augment-Tool-Choice-Demotion boolean bundle Demotes tool_choice: "required" to "auto" once a tool has dispatched. Off: required is honored on every turn.
X-Erebine-Augment-Reasoning-Normalization boolean bundle Moves inline <think> bodies (balanced, unclosed, or orphan closes) out of the visible content and into the reasoning channel: message.reasoning_content on the response and reasoning_content on each streaming delta. Off (the faithful floor): the backend's bytes pass through untouched, tags and all; if the backend delivered a separate reasoning channel it is re-inlined as a <think>...</think> block ahead of the answer so billed reasoning tokens are never dropped.
X-Erebine-Augment-Tool-Top-K boolean off Narrows your tools array to a cosine-ranked top-K view of the turn, using the workspace's embedding endpoint and embedding_top_k preference (default 8). Always-on tools and any function named by tool_choice are protected. Inert unless the request binds a workspace with an embedding endpoint configured; with nothing to rank by, your tools pass through untouched. Off by default everywhere -- including first-party origins and under X-Erebine-Augment -- because dropping a tool you declared is an augmentation, not transport translation. This header is the only switch that turns it on.
X-Erebine-Augment-Media-URL-Resolution boolean on Fetches a remote https image URL and inlines it as a data: URL before dispatch. Client-supplied data: URLs are validated but passed through byte-identical either way; this governs only the remote-fetch leg. Off: an unfetchable remote URL is a 400. On by default everywhere, including at the faithful floor and under X-Erebine-Augment: off -- the canonical OpenAI vision request sends image_url and expects the server to fetch it. This header is the opt-out. See Passthrough and Augmentation.
X-Erepress boolean off for API traffic Lossless in-path token compression. Erepress's own header, and the per-feature override for that feature: it is not named X-Erebine-Augment-* but it obeys the same precedence. See Erepress.
X-Erebine-Vendor-Events boolean off Adds Erebine's vendor SSE events (x_-prefixed) to a streaming response. Off: the stream carries only spec-shaped chunks and [DONE], which is what a strict OpenAI client requires.
X-Conversation-Id string derived from the conversation Affinity key for the Semantic Router: it holds a multi-turn thread on one endpoint. Send the same value on every turn of a conversation. Absent, the key is derived from your first textful user turn plus the opening of the first turn after it that is neither system nor developer; system and developer turns are never folded in. That is stable for a client that replays its thread verbatim. Ignored when you address an endpoint slug directly.
X-Workspace-Id ws_ id or UUID project default workspace Scopes the request to a workspace. A workspace in the URL path wins over this header. Must belong to your project, and to your API key's workspace allowlist when it has one.
X-Erebine-Workspace ws_ id or UUID session-pinned workspace Workspace scoping on the MCP endpoint. Same rule, different surface: X-Workspace-Id scopes inference, this scopes MCP.
X-Chat-Id chat UUID none Names the chat whose saved tool scope X-Erebine-Augment-Tool-Allowlist intersects. Does nothing on its own.
X-SLO-TTFT-Ms positive number none Target time-to-first-token in milliseconds. The router prefers workers likely to meet it. Invalid values are ignored.
X-SLO-TPOT-Ms positive number none Target time-per-output-token in milliseconds. Same routing preference, for throughput.
traceparent W3C trace context none Continues your trace through the router and the worker. Absent, the router starts its own. A malformed value is discarded, not an error.

Response Headers

Header Value When What it tells you
X-Request-ID request id every response Matches the response body id where the body has one; otherwise a per-request correlation id. Response-only: the server never reads it from your request. Quote it in a support ticket.
X-Erebine-Worker-ID worker id non-streaming chat completions, Responses, embeddings, rerank, and score Which worker answered. Correlates latency with routing. Absent on streaming responses (the head is written before a worker is chosen), on responses no worker served, and on audio transcription and translation (not yet covered).
X-Erebine-Routed-Model model name semantic-routed responses The model that actually served the request. The body echoes the model string you sent, so this header is how you learn what answered.
X-Erebine-Routed-Endpoint endpoint slug semantic-routed responses The endpoint the router dispatched to. Sent alongside X-Erebine-Routed-Model.
X-Erepress applied when compression ran Erepress compressed this turn's messages. Absent means it did not run.
X-Erebine-Budget-Warning <level>:<window> near a budget cap A configured token budget is close to its cap, e.g. workspace:daily. The request was still served in full.
X-RateLimit-Limit number rate-limited routes Request quota for the current window. Also sent as RateLimit-Limit.
X-RateLimit-Remaining number rate-limited routes Requests left in the current window. Also sent as RateLimit-Remaining.
X-RateLimit-Reset seconds rate-limited routes Seconds until the window resets. Also sent as RateLimit-Reset.
X-RateLimit-Warning approaching_limit under 20% of quota left Back off before you are throttled.
Retry-After seconds 429 and retryable errors Standard HTTP. Wait this long before retrying. See Error Handling.

Other Surfaces

Headers on the endpoints outside the OpenAI-compatible surface.

Header Direction What it does
X-Part-Number request Chunk index on a chunked model upload. Interchangeable with the part_number query parameter. See Model Upload.
X-Chunk-Checksum request SHA256 hex digest of the chunk body. Required on a chunk upload; a mismatch rejects the chunk.
X-Confirm-Project-Name request Required on project deletion through the management API. Must match the project's name exactly, or the delete is refused.

erebine Extensions

Envelope fields and behaviors beyond the OpenAI spec. Headers are in the Header Reference.

Response Fields Beyond OpenAI

Field Description
x_adjusted_reasoning_effort Resolved reasoning effort after model-family clamping. Present only when the requested reasoning_effort is not supported by the model family and the router remaps it to a supported level (e.g. high resolves to medium on a family that lacks high); omitted when the requested effort is already supported or none was set.
usage.prompt_tokens_details.cached_tokens Prefix-cache hits served for this request. Same field name as OpenAI; populated for every endpoint, not just specific model families.

Error Envelopes

Erebine error responses follow the OpenAI { "error": { ... } } envelope but extend it in two ways that SDK clients switching on error.type must handle:

  • Non-spec type values are emitted, including authorization_error, internal_error, validation_error, service_error, stream_error, forbidden_error, and insufficient_quota. Treat unknown type values defensively rather than asserting against the OpenAI enum.
  • Additional envelope keys retry_after (seconds) and retry_strategy (e.g. exponential) accompany retryable failures and should be preferred over a fixed backoff.

See Error Handling for the full taxonomy.

Cancellation

Cancel an in-flight streaming completion by issuing POST /{project_id}/{endpoint_slug}/v1/chat/completions/{id} using the completion id returned in the first SSE chunk.

Score / Rerank

The reranking endpoint (POST /v1/score) is not exposed through the OpenAI SDK surface. See Rerank API for the raw HTTP shape and examples.

SDK Quick Start

Pick a language. The tabs switch in sync.

Basic Request

Python
from openai import OpenAI client = OpenAI( base_url="https://api.erebine.ai/proj_ABC123/my-endpoint/v1", api_key="ere_YOUR_PROJECT_SLUG_YOUR_API_KEY" ) response = client.chat.completions.create( model="YOUR_MODEL_NAME", messages=[{"role": "user", "content": "Hello!"}] ) print(response.choices[0].message.content) print(f"Service tier: {response.service_tier}") print(f"System fingerprint: {response.system_fingerprint}") if response.usage.prompt_tokens_details: print(f"Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}") if response.usage.completion_tokens_details: print(f"Reasoning tokens: {response.usage.completion_tokens_details.reasoning_tokens}") if response.choices[0].message.refusal: print(f"Refusal: {response.choices[0].message.refusal}")
Node.js
import OpenAI from 'openai'; const client = new OpenAI({ baseURL: 'https://api.erebine.ai/proj_ABC123/my-endpoint/v1', apiKey: 'ere_YOUR_PROJECT_SLUG_YOUR_API_KEY' }); const response = await client.chat.completions.create({ model: 'YOUR_MODEL_NAME', messages: [{ role: 'user', content: 'Hello!' }] }); console.log(response.choices[0].message.content); console.log(`Service tier: ${response.service_tier}`); console.log(`System fingerprint: ${response.system_fingerprint}`); console.log(`Cached tokens: ${response.usage?.prompt_tokens_details?.cached_tokens}`); console.log(`Reasoning tokens: ${response.usage?.completion_tokens_details?.reasoning_tokens}`);
Go
package main import ( "bytes" "encoding/json" "fmt" "io" "net/http" ) func main() { body := map[string]interface{}{ "model": "YOUR_MODEL_NAME", "messages": []map[string]string{ {"role": "user", "content": "Hello!"}, }, } jsonBody, _ := json.Marshal(body) req, _ := http.NewRequest("POST", "https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions", bytes.NewReader(jsonBody)) req.Header.Set("Authorization", "Bearer ere_YOUR_PROJECT_SLUG_YOUR_API_KEY") req.Header.Set("Content-Type", "application/json") resp, err := http.DefaultClient.Do(req) if err != nil { panic(err) } defer resp.Body.Close() data, _ := io.ReadAll(resp.Body) fmt.Println(string(data)) fmt.Println("Request ID:", resp.Header.Get("X-Request-ID")) fmt.Println("Worker ID:", resp.Header.Get("X-Erebine-Worker-ID")) }
curl
curl https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions \ -H "Authorization: Bearer ere_YOUR_PROJECT_SLUG_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "YOUR_MODEL_NAME", "messages": [{"role": "user", "content": "Hello!"}] }'

SLO Headers

Set per-request latency targets that influence routing. Optional on every call.

Python
response = client.chat.completions.create( model="YOUR_MODEL_NAME", messages=[{"role": "user", "content": "Hello!"}], extra_headers={ "X-SLO-TTFT-Ms": "500", "X-SLO-TPOT-Ms": "50" } )
Node.js
const response = await client.chat.completions.create({ model: 'YOUR_MODEL_NAME', messages: [{ role: 'user', content: 'Hello!' }] }, { headers: { 'X-SLO-TTFT-Ms': '500', 'X-SLO-TPOT-Ms': '50' } });
curl
curl https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions \ -H "Authorization: Bearer ere_YOUR_PROJECT_SLUG_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -H "X-SLO-TTFT-Ms: 500" \ -H "X-SLO-TPOT-Ms: 50" \ -d '{ "model": "YOUR_MODEL_NAME", "messages": [{"role": "user", "content": "Hello!"}] }'

Streaming

Set stream: true. Each event is a line prefixed with data: carrying a JSON chunk; the stream terminates with a literal data: [DONE]. When stream_options.include_usage is true, the final pre-[DONE] chunk carries a populated usage object. See Streaming API for the parsing patterns and the two supported wire shapes.

curl
curl https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions \ -H "Authorization: Bearer ere_YOUR_PROJECT_SLUG_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -N \ -d '{ "model": "YOUR_MODEL_NAME", "messages": [{"role": "user", "content": "Write a poem about AI"}], "stream": true }'

Inspect Response Headers

curl
curl -v https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions \ -H "Authorization: Bearer ere_YOUR_PROJECT_SLUG_YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "YOUR_MODEL_NAME", "messages": [{"role": "user", "content": "Hello!"}] }' 2>&1 | grep -i "x-request-id\|x-erebine\|x-ratelimit"

Typed Response Parsing (Go)

For typed access to service_tier, system_fingerprint, and usage:

Go
type Usage struct { PromptTokens int `json:"prompt_tokens"` CompletionTokens int `json:"completion_tokens"` TotalTokens int `json:"total_tokens"` } type ChatResponse struct { ID string `json:"id"` Model string `json:"model"` ServiceTier string `json:"service_tier"` SystemFingerprint string `json:"system_fingerprint"` Usage Usage `json:"usage"` Choices []struct { Message struct { Role string `json:"role"` Content string `json:"content"` } `json:"message"` FinishReason string `json:"finish_reason"` } `json:"choices"` } var parsed ChatResponse if err := json.Unmarshal(data, &parsed); err != nil { panic(err) } fmt.Println("Tier:", parsed.ServiceTier, "Tokens:", parsed.Usage.TotalTokens)

LangChain

LangChain reaches an OpenAI-compatible endpoint through ChatOpenAI. Install langchain-openai, point base_url at the endpoint slug.

Python
from langchain_openai import ChatOpenAI llm = ChatOpenAI( base_url="https://api.erebine.ai/proj_ABC123/my-endpoint/v1", api_key="ere_YOUR_PROJECT_SLUG_YOUR_API_KEY", model="YOUR_MODEL_NAME" ) response = llm.invoke("What is the capital of France?") print(response.content) # Streaming for chunk in llm.stream("Write a poem about AI"): print(chunk.content, end="")

LlamaIndex

LlamaIndex routes through its OpenAI LLM class. Install llama-index-llms-openai.

Python
from llama_index.llms.openai import OpenAI llm = OpenAI( api_base="https://api.erebine.ai/proj_ABC123/my-endpoint/v1", api_key="ere_YOUR_PROJECT_SLUG_YOUR_API_KEY", model="YOUR_MODEL_NAME" ) response = llm.complete("What is the capital of France?") print(response.text)

Full Parameter Parity

Every OpenAI Chat Completions parameter listed below is accepted with the same semantics as the upstream spec. The footnote column flags the small set with Erebine-specific notes; everything else passes through unchanged.

Supported request parameters (27)
Parameter Notes
model Model name as configured on your endpoint.
messages System, user, assistant, tool, and developer roles.
max_tokens Honored verbatim. 400 context_length_exceeded if it does not fit the context window, unless clamping is opted in.
max_completion_tokens Preferred over max_tokens. Same context-window behavior.
temperature 0.0 to 2.0.
top_p Nucleus sampling.
stream SSE streaming. See Streaming API.
stream_options Set include_usage: true for token usage in the final chunk.
stop String or array of strings.
tools See Tool Calling.
tool_choice auto, none, required, or specific function.
parallel_tool_calls Parallel tool calls in a single response.
logprobs See API Reference.
top_logprobs 0-20, engine-enforced cap. Requires logprobs: true.
reasoning_effort "low", "medium", or "high". May be clamped per model; resolved value surfaces as x_adjusted_reasoning_effort.
prediction Speculative decoding. See Predicted Outputs.
service_tier Influences routing priority and billing within the endpoint tier. Does not override the endpoint tier itself. See Service Tiers.
seed Use with system_fingerprint for reproducibility.
n 1-128. Router-side fan-out emits multiple choices even in streaming mode.
frequency_penalty -2.0 to 2.0.
presence_penalty -2.0 to 2.0.
logit_bias Token-id map, -100 to 100.
response_format text, json_object, or json_schema.
metadata Up to 16 key-value pairs.
user End-user identifier for abuse monitoring.
web_search_options Enable in-line web search. Populates message.annotations with URL citations.
store Retrieve later via GET /{project_id}/{endpoint_slug}/v1/chat/completions/{id}.
Supported response fields
Field Description
service_tier Present in every response and SSE chunk. Value is the Erebine endpoint tier slug, not the OpenAI vocabulary.
system_fingerprint Backend configuration identifier for reproducibility tracking.
message.refusal Refusal text when the model declines. SSE delta.refusal coverage on the chat-completions path is sparse; prefer the non-streamed message.refusal.
message.annotations URL citations when web_search_options is set. Defaults to an empty array. Also streams via delta.annotations.
logprobs Per-token log probabilities with content and refusal arrays, including top_logprobs.
usage.prompt_tokens_details Includes cached_tokens served from prefix cache.
usage.completion_tokens_details Includes reasoning_tokens, accepted_prediction_tokens, rejected_prediction_tokens.
x_adjusted_reasoning_effort Erebine extension. Resolved reasoning effort after model-family clamping.

Keyboard shortcut: press Shift+C while a code block is focused to copy it. Cmd+Shift+C and Ctrl+Shift+C copy the nearest visible block from anywhere on the page.