Chat Completions

Chat Completions

Creates a model response for the given chat conversation. This is the primary endpoint for interacting with language models.

Create Chat Completion

POST /:project_id/:endpoint_slug/v1/chat/completions

URL path: :project_id is your project's external ID (e.g., proj_ABC123) and :endpoint_slug is the slug of the endpoint to route through (e.g., my-endpoint). Your full base URL is https://api.erebine.ai/proj_ABC123/<endpoint-slug>/v1.

Request Body

Optional Request Headers

Every header Erebine reads or sends, request and response, is listed in the Header Reference.

Response Headers

Message Object

Multimodal Input

A message's content array carries typed parts. Text rides text parts, image input rides image_url parts, and audio input rides input_audio parts. All three are the OpenAI content-part shapes, so an unmodified OpenAI client sends them as-is. A part is served only when the resolved model advertises the matching input modality (vision for images, audio for input_audio); audio input is operator-gated and on by default. See Audio Input for the input_audio shape.

video_url is a non-standard vLLM extension, not part of the OpenAI specification, and is off by default. It is served only when the operator enables video input (EREBINE_VIDEO_INPUT) and the resolved model advertises video_input; otherwise a video_url part returns 400. The faithful /v1 surface never advertises video as an OpenAI-standard input.

Example Request

Log Probabilities Response

When logprobs: true is set in the request, each choice includes a logprobs object with per-token log probabilities. The top_logprobs parameter controls how many alternative tokens are returned (0-20).

Usage Object

Response Message Fields

The message object in each response choice contains the assistant's output. In addition to role and content, the following fields may be present:

Additional Response Fields

Passthrough and Augmentation

Erebine is an OpenAI drop-in, and it behaves like one by default. A request that carries no Erebine headers is served exactly as it was sent:

• Your messages array reaches the model unchanged. Nothing is prepended, inserted, rewritten, or dropped.
• max_tokens, tools, tool_choice, and response_format are honored verbatim.
• No tool call is executed that the model did not emit.
• One client request is one model turn. You are billed for that turn and no other.
• Where the request cannot be served as specified, you get an error, not a substitute answer. See Error Semantics.

Erebine's agentic harness -- parallel-tool-call nudging, corrective retries, research grounding and fallback search, context compaction, in-path token compression -- is opt-in. Ask for it and you get it, per request, in whole or by the feature.

Under the harness, a corrective retry is an additional billed model turn: one client request can drive several dispatches. Your max_tokens still bounds the total -- it caps the cumulative output across the whole request, not each turn -- so once the run's generated output reaches max_tokens the loop stops and the response closes with finish_reason: "length". At the faithful floor (the default for API traffic) none of this applies: one request is one turn, billed once.

Turn the Harness On

One header, every feature. Send off (the default for API traffic) and none of them run.

Augmentation Headers

Twelve per-feature X-Erebine-Augment-* overrides turn the harness on or off one feature at a time -- the max_tokens floor, context compaction, the output-token clamp, capability tool-drop, corrective retries, the fan-out nudge, fabricated tool calls, the chat tool allowlist, tool-choice demotion, reasoning-channel normalization, tool top-K, and media-URL resolution -- and X-Erepress is the thirteenth, for in-path token compression (Erepress). One header, X-Erebine-Augment-Tool-Top-K, is off by default everywhere -- including first-party origins and under X-Erebine-Augment -- and is the only switch that turns it on, because narrowing your declared tools is an augmentation, not transport translation. One header runs the other way: X-Erebine-Augment-Media-URL-Resolution is on everywhere, including at the faithful floor, because fetching a remote image_url server-side is what the OpenAI vision request shape already means; : off is the way to opt out. Every remaining header follows the bundle default: off at the faithful floor, on for first-party origins, and toggleable either way per request. Each header, what it does when it is on, and what you get when it is off: Header Reference.

One reachability note: the fan-out nudge (X-Erebine-Augment-Fanout-Nudge) fires only when a turn advertises two or more server-owned tools (the built-in research tools, or workspace / Responses server tools). A request whose tools are all your own client-side functions -- the usual chat-completions case -- never triggers it, so the header is a safe no-op there. Erebine always honors the header; it never rejects it for being unreachable.

Values

Every one of these headers takes on, 1, true, or yes to enable, and off, 0, false, or no to disable. Values are case-insensitive. A value that is none of these is treated as if the header were absent, so a typo never flips a feature. This is the same vocabulary every boolean header on the API speaks.

Precedence

1. A per-feature header wins.
2. Otherwise X-Erebine-Augment decides.
3. Otherwise the default applies: off for API traffic, on for Erebine's own chat and MCP surfaces.

So X-Erebine-Augment: on plus X-Erebine-Augment-Compaction: off runs the whole harness except compaction, and a context overflow still returns a 400.

Error Semantics

Faithful mode has no repair path, so a request that cannot be served as specified fails with a spec-shaped error instead of quietly becoming a different request. These are guarantees, not regressions: the answer you get is always an answer to the question you asked.

Both conditions are recoverable client-side: shorten the input, lower max_tokens, route to a tool-capable model -- or opt into the matching augmentation and let Erebine handle it.

Predicted Outputs

The prediction parameter enables speculative decoding: you supply a predicted output and the model verifies it in parallel rather than generating each token sequentially. When the prediction matches, generation is significantly faster. When it does not match, the model falls back to normal generation.

Prediction Object

Example Request

Response Token Details

When a prediction is provided, the response usage.completion_tokens_details includes prediction-specific token counts:

• accepted_prediction_tokens, Tokens from your prediction that the model verified and used. Higher values indicate a better prediction.
• rejected_prediction_tokens, Tokens from your prediction that the model discarded and regenerated. These still count toward billing.

Best Practices

• Use predictions for code editing, document reformatting, and template-based generation where you can anticipate the output structure.
• Accurate predictions reduce latency via parallel verification. Inaccurate predictions may be slower than no prediction at all.
• Monitor accepted_prediction_tokens vs rejected_prediction_tokens to evaluate prediction quality.

Tool Calling

Erebine supports OpenAI-compatible function calling. Define tools in your request and the model may generate tool calls in its response.

Tool Definition

Tool Call Response

When the model decides to call a tool, the response includes tool_calls instead of text content:

Tool Result Message

Send the tool result back using role tool with the matching tool_call_id:

Semantic Router

The semantic router picks the generative endpoint that best matches each request, so a single base URL serves your whole workspace. Point an OpenAI-compatible client at it and send chat completions as usual; the router reads the request and dispatches it to the endpoint that fits.

This section covers the route itself. How a Turn Is Routed covers what decides the endpoint, including operator rules and the workspace routing mode; Endpoint Eligibility covers which endpoints are candidates in the first place; Multi-Model Turns covers the decomposing variant of this route; and the Semantic Router Management API covers configuring all of it.

Create Chat Completion (routed)

POST /:project_id/_semantic/router/v1/chat/completions

URL path: :project_id is your project's external ID (e.g., proj_ABC123). Unlike the per-endpoint path, there is no endpoint slug; the router selects the endpoint for you. The request and response bodies are the standard Chat Completions shapes.

Base URL

To pin a request to a specific workspace, add the workspace ID (a ws_-prefixed value, e.g. ws_9f2c8a) as a path prefix:

Automatic model selection

The router chooses the endpoint, so you do not pin a model. The model field in the request body is advisory and any non-empty string is accepted; the model that actually served the request is reported on the X-Erebine-Routed-Model response header.

How it works. Every generative endpoint in a workspace carries a short profile of what it is best at, drawn from its model and a routing description you can edit in workspace settings. For each request, the router compares the intent of the prompt against those profiles and dispatches it to the closest fit. When no endpoint clears the workspace's match threshold, the request goes to your configured fallback, or a deterministic default, so a call never stalls waiting for a perfect match. The routing decision itself is not billed: only the completion that serves the request is metered, exactly as a direct call would be.

Point one base URL at a workspace and let it place traffic across your endpoints. Tune the behavior per workspace from the Semantic Router settings: turn routing on or off, set the match threshold, pin a fallback, or take an endpoint out of the routing path. Reach for it whenever a workspace runs more than one generative endpoint and you want each request routed on merit instead of naming an endpoint slug yourself.

Conversation routing. The router reads the whole thread, not just your latest message, so a short follow-up still routes by what the conversation is about. A conversation sticks to the endpoint it landed on while that endpoint stays the best fit, and only moves when the topic clearly shifts. Stickiness is keyed on the conversation; see Pin a Conversation for how the key is decided and how to control it.

Which Model Served the Request

The router re-picks an endpoint on every request, so the serving model can change from one turn of a conversation to the next. The response body is not where you find that out: the model field echoes the string you sent, because rewriting your request body is not something the router does. The served model is reported out of band, on two response headers:

Both are set on every request this base URL dispatches, including the ones that fall through to your configured fallback, and on streaming responses they are committed before the first SSE byte -- so a streaming client reads them on the same turn, not after it. If you log one thing about a routed request, log these.

Pin a Conversation

Send a stable X-Conversation-Id for the life of a conversation. That value is the affinity key: it is what holds a multi-turn thread on one endpoint instead of re-deciding it per turn. This matters most for tool calling, where a swap mid-thread hands your tool-call history to a model with a different tool-call dialect.

Any string works. It is opaque to the router: use your own thread ID, a UUID, whatever you already key the conversation on. Use a new value when the conversation is new.

Without the header the router derives a key from the conversation's own turns instead -- the first user message carrying text, plus the opening of the first turn after it that is neither system nor developer (the assistant reply, or a tool result) -- so an OpenAI-compatible client that replays its history verbatim on each turn gets stickiness for free. System and developer turns are never folded in: they mutate between turns of the same conversation, and hashing them would mint a new key every turn. That derivation is only as stable as those two turns. A client that trims its context head, or edits the first user message or the first reply, changes those bytes and mints a new key: the conversation loses its affinity and can be routed to a different model without anything in the response saying so. If your client does any of that, send the header.

Example Request

How a Turn Is Routed

Four things can decide where a generative turn runs, and they are consulted in a fixed order. The first one that answers wins.

1. An operator routing rule. Rules are evaluated before anything else, including the workspace's routing mode and the project's semantic-router switch. A matching rule dispatches to the endpoint it names.
2. A pinned endpoint. A pin on the chat or on the workspace wins over every routing mode. This is the guarantee: a pin always wins.
3. The project's semantic-router switch. With semantic_router_enabled explicitly false, the workspace routes concretely no matter what mode it recorded. Operator policy outranks the workspace's choice.
4. The workspace's routing mode, and failing that, the default its cohort was created with.

Workspace Routing Mode

A workspace records how it wants generative turns routed, on the field inference_routing_mode. It is readable and writable on the workspaces API and through erectl workspaces.

On a PATCH, omitting the field leaves the recorded choice alone, a string sets it, and an explicit null clears it. Any other value is rejected with 400. Setting "semantic", or clearing back to null, also drops the workspace's pinned inference endpoint -- a pin outranks the mode, so leaving one in place would record a choice that could never take effect.

Workspaces created from this release on default to semantic routing. Existing workspaces are never flipped. The default is stamped at creation, so nothing already running changes. A workspace created today inside an older project gets the new default; the stamp follows creation time, not the parent. An explicitly chosen mode is inherited by child workspaces.

What semantic routing needs

Semantic routing engages when all of the following hold. Note the second one: an embedding endpoint is not required unconditionally, only when there is more than one candidate to choose between.

• The workspace has at least one active generative endpoint.
• If it has two or more, the project also has an embedding endpoint. With exactly one generative endpoint the router short-circuits straight to it and needs no embedding endpoint at all.
• The project's semantic_router_enabled setting is not explicitly false. It fails open: a missing or unreadable value reads as enabled.

An operator routing rule bypasses all three, because it names an endpoint outright and nothing needs classifying.

Operator Routing Rules

A routing rule short-circuits routing: when one fires, no classification runs and the request goes straight to the endpoint the rule names. A rule can match on the query text (query_contains), on the estimated input size (min_input_tokens), on the model string the caller sent (exact_request_model), or on any combination of the three. Rules are evaluated on the semantic routes only -- _semantic/router/v1/chat/completions, _semantic/router/v1/responses, and the multi-model path. Addressing an endpoint slug directly bypasses them, as it bypasses routing generally.

Evaluation order. Rules run in descending priority; equal priorities break by stored order, earliest first. Disabled rules are skipped before any condition is tested. The first rule whose every condition matches wins and evaluation stops -- later rules, including higher-specificity ones, are never consulted.

Match conditions

A rule matches only when all of its conditions hold.

Action

Rules also carry name, an integer priority (default 0), and a boolean enabled (default true). Author them from the Semantic Router settings page, from erectl semantic-router rules, or over the management API.

Region Locality

An endpoint with serving capacity in the router's own region earns a small ranking preference, controlled by semantic_router_proximity_bonus (default 0.02, range 0 to 1 inclusive).

It is a tie-breaker, not an override, and the three properties that make it safe are worth knowing:

• Rank-only. The match threshold is applied to the unmodified score, so the bonus can reorder candidates that already qualify but can never lift one that does not.
• Symmetric. An incumbent endpoint is ranked with its own bonus exactly as a challenger is, so a conversation will not drift to a different model just because one side is local.
• Off by setting it to zero. Zero is a real value meaning "no locality preference", not an unset one.

Region names are compared exactly, including case. Locality applies on the chat-completions and Responses semantic routes; the multi-model path does not use it.

Endpoint Eligibility

Before ranking, the router removes endpoints that cannot serve the request as written. Four filters run, in this order: modality (images, video, audio input), context window (estimated input plus your max_tokens), tool calling (a request declaring tools), and structured output (a request naming response_format).

Modality

A turn carrying images, video, or audio input requires an endpoint whose model advertises the matching capability. A request whose modality no endpoint in the project can serve is rejected rather than answered by a model that will ignore the attachment.

Context window

An endpoint is eligible when its context window fits the request's estimated input plus the output budget the caller asked for. The output budget is your max_tokens (or max_completion_tokens) when you send one, and a conservative reserve of 2048 tokens when you do not. The input side is deliberately over-estimated so a near-miss errs toward the larger endpoint.

Two consequences worth planning around:

• Models with a context window under 4096 tokens are routable. The requirement is computed from the actual request, not from a fixed floor, so a small-context endpoint is eligible for a small request.
• An explicitly large max_tokens narrows the candidate set. Sending max_tokens: 32000 makes every endpoint that cannot reserve 32000 output tokens ineligible, even for a one-line prompt. Clients that hardcode a large default will see a smaller pool than they expect.

When the workspace had candidates and this filter removed all of them, the request returns 400 with code: "context_length_exceeded". When the workspace had no routable endpoints to begin with, it returns 404 with code: "no_routable_endpoints" -- a different problem with a different fix.

Tool calling

A request that declares tools, or carries tool-call history, is narrowed to endpoints whose model supports tool calling.

Structured output

A request naming response_format with json_object or json_schema is narrowed to endpoints known to honour a JSON schema. This matters more than most capability checks: a model that ignores a schema does not error, it returns plausible prose, and the failure surfaces in your parser rather than in the response.

Multi-Model Turns

POST /:project_id/_semantic/router/v1/chat/completions/multimodel

This route decomposes a turn into sub-tasks, dispatches each to the endpoint best suited to it, and merges the results. Request and response bodies are the standard Chat Completions shapes. The workspace prefix and the X-Workspace-Id header work exactly as they do on the single-model semantic route.

Turn it off for a project by setting semantic_router_multi_model_enabled to false; the route then degrades to ordinary single-endpoint dispatch rather than erroring.

Cost of n greater than 1

For a plan of S sub-tasks, a turn requesting n choices costs n x S upstream dispatches, plus one synthesis dispatch per choice when the plan has three or more sub-tasks. Previously only the first choice ran the plan; the remaining choices ran a plain single-model completion that answered a weaker question while still being billed through this route.

Treat those as floors. A sub-task whose model emits tool calls costs more than one dispatch, and an unusually large combined result can add the synthesis pass to a plan of fewer than three sub-tasks. n is not capped on this route. A turn that sets n > 1 and asks for research is rejected with 400, code: "unsupported_value", param: "n".

Plans of three or more steps

A plan of three or more sub-tasks runs a real synthesis pass over the step results, and the reported usage covers both the steps and the synthesis. Smaller plans return the terminal step's own answer. Internal merge scaffolding never appears in an assistant message on either path.

Errors

A sub-task that needs a modality no endpoint in the project can serve fails the whole turn rather than quietly handing the work to a text-only model:

A turn that fails outright surfaces the upstream error instead of returning 200 with an empty body. On a buffered request that is an ordinary error response carrying the upstream code -- a dispatch failure with no equivalent client-facing code reports 500 internal_error, while a recognized upstream condition keeps its own mapping (for example context_length_exceeded stays 400). On a streaming request the status line is already committed, so the same error envelope arrives as a data: frame followed by [DONE]; with n > 1 that frame is written once, after the last choice closes.

Semantic Router Management API

Read and write a project's semantic-router configuration, manage its routing rules, and read routing telemetry. This is the surface erectl semantic-router drives; use the CLI unless you are building automation.

Authorization

Every route takes Authorization: Bearer <key> with a key carrying the management scope. A key without it returns 403 with code: "scope_insufficient"; a key bound to a different project returns 403 with code: "project_mismatch". :project_id accepts either the project UUID or its external ID.

There is no owner-versus-operator distinction on these routes. Any holder of a management-scoped key can write the project's routing configuration and rules, where the dashboard reserves the equivalent write to a project owner. Two consequences follow, and both are reasons to scope keys narrowly: see Route Decisions for the read side, which is broader still.

Routes

Request bodies are capped at 16 KiB. These routes are subject to the Management API rate limit.

Semantic Router Settings

PATCH .../config takes a flat JSON object of settings to change. Keys you omit are left alone; an explicit null clears a key back to the deployment default. A key outside this table is rejected with 400 unknown_semantic_router_setting, so a management key cannot reach unrelated project preferences through this endpoint.

A number outside 0 to 1 is rejected, not clamped: the router discards a value it cannot use and falls back to the deployment default, so an out-of-range setting would store cleanly and then change nothing. Likewise a boolean key must be a JSON boolean -- the string "false" is rejected rather than read as enabled.

GET .../config returns those settings under preferences, alongside the routable and referenced-but-unavailable endpoint lists, whether an embedding endpoint is present, the resolved availability, the deployment defaults, a 7-day usage roll-up, and the same routing_health block the health route returns.

Managing Rules Over the API

Rule request bodies are flat; rule responses are nested. A POST or PATCH takes the match and action fields at the top level -- name, query_contains, min_input_tokens, exact_request_model, endpoint_id, fallback_to_classification, priority, enabled -- and the response returns them grouped under match and action. See Operator Routing Rules for what each field means.

id is minted by the server and is not accepted on create. On a PATCH, an omitted field means "leave it alone" -- there is no way to clear min_input_tokens or exact_request_model, so a rule that must lose a condition has to be deleted and recreated. query_contains replaces the stored array whole.

Validation

Rules are validated before the write, not rejected after it. A project may hold at most 20 rules; a rule may carry at most 10 query_contains entries of at most 200 characters each. endpoint_id must name a functional generative endpoint in the project -- a suspended endpoint is still a legal target. A rule with an empty query_contains and no min_input_tokens or exact_request_model is rejected: an all-of over an empty list is vacuously true, so it would capture every request in the project.

The rules are stored as one array and revalidated whole on every write, so a pre-existing rule whose target endpoint has since been deleted will block an edit to an unrelated rule. The error names the offender in its param field, indexed by position in GET .../rules. Deleting the blocking rule always succeeds, so this cannot wedge a project.

GET .../rules returns rules in stored order, which is not evaluation order. Read priority to predict which rule fires; stored order matters only as the index in the error above.

Routing Health

GET .../health returns counts and totals over a fixed 24-hour window and reads no prompt content, which makes it the right thing to poll and the right thing to paste into a ticket.

fallback_pct is deliberately taken over classified decisions only. Single-endpoint and multi-model turns are decided before any classification happens, and folding them in would drive the ratio toward zero while saying nothing about how classification is going. The residue -- decision_count minus classified_count minus rule_count -- is exactly those turns.

Health is telemetry, so a failure to compute it returns zeros rather than failing the request.

Route Decisions

GET /:project_id/v1/management/semantic-router/decisions

Answers "why did this request go to that endpoint". Rows cover a fixed 24-hour window, newest first. limit defaults to 50 and is clamped to 1-200; a non-numeric value is rejected with 400 invalid_limit rather than quietly defaulted. There is no time or workspace parameter.

Response

Decision rows are retained for a short operational window and are removed with a project's content when the project is purged.

Audio and Realtime

Speech-to-text, text-to-speech, and streaming transcription, all on the OpenAI shapes. Full request and response detail, form fields, and error codes are on the Audio page; the routes are:

Create Transcription

POST /:project_id/:endpoint_slug/v1/audio/transcriptions

Transcribes an uploaded audio file in its own language. Multipart upload, served by a transcription endpoint. See Audio.

Create Translation

POST /:project_id/:endpoint_slug/v1/audio/translations

Transcribes an uploaded audio file and translates the transcript to English. See Translations.

Create Speech

POST /:project_id/:endpoint_slug/v1/audio/speech

Synthesizes audio from text. JSON body, served by a speech endpoint; returns audio bytes or a Server-Sent Events audio stream. See Speech.

Realtime Transcription

WS /:project_id/:endpoint_slug/v1/realtime

Streams speech to text over a WebSocket, the transcription subset of the OpenAI Realtime API. Streaming ASR only, not speech-to-speech. Requires a realtime endpoint. See Realtime Transcription.

Rate Limits

Every API response includes rate limit headers. Rate limits are enforced per service tier using a sliding window algorithm.

Rate Limit Response Headers

Both standard (draft IETF) and X- prefixed headers are returned for broad client compatibility:

Rate Limits by Tier

Burst capacity allows short-term traffic spikes above the base limit without immediately blocking requests.

The Requests/Min column is the tier's base rate. The RateLimit-Limit / X-RateLimit-Limit header advertises the enforced ceiling = base + burst, which is the point at which a 429 is actually returned. The effective ceilings are therefore free 96 (64 + 32), cpu 192 (128 + 64), and gpu 384 (256 + 128). A client that reads "free = 64/min" and receives X-RateLimit-Limit: 96 is seeing the base rate and the enforced ceiling respectively -- there is no discrepancy. Self-hosted is unmetered and omits the numeric rate-limit headers entirely.

429 Rate Limit Exceeded

When the rate limit is exceeded, the API returns a 429 Too Many Requests response with retry guidance:

The Retry-After header is also set to the same value. The retry_after value is the number of seconds until a slot frees for your request; clients should back off at least that long and add exponential backoff with jitter across successive 429s.

Error envelope type and code values shown here illustrate intent; the canonical list of error types and their HTTP status mapping is documented on the Errors page. Treat that page as the source of truth if a value differs.

Management API Rate Limits

Management API endpoints (batch, conversations, files, uploads, webhooks, and SLOs) are subject to a separate per-project rate limit, distinct from the inference rate limit -- management traffic does not consume your inference budget, and inference traffic does not consume the management limit. Most management (control-plane) operations share a 60 rpm per-project window. Two high-volume data-plane routes carry their own dedicated, higher per-project window so bulk workloads are not throttled by the control-plane limit:

• File upload and content download (POST /v1/files, GET /v1/files/{id}/content): default 600 rpm, configurable via EREBINE_FILES_API_RATE_LIMIT_RPM.
• Conversation item append (POST /v1/conversations/{id}/items): default 600 rpm, configurable via EREBINE_CONVERSATIONS_API_RATE_LIMIT_RPM.

Response headers on management API endpoints:

Both the IETF draft form (RateLimit-*) and the vendor form (X-RateLimit-*) are returned, matching the inference surface. When the limit is exceeded, the API returns HTTP 429 Too Many Requests. Inference endpoints (chat completions, embeddings, responses) have separate per-endpoint rate limits as described above and are not affected by the management API rate limit.

Request Timeouts

Request timeouts are determined by your endpoint's service tier. These are not client-configurable.

Request Deadline Timeout

The maximum time a request can wait for a response before being cancelled:

Idle Stream Timeout

For streaming requests, the maximum time between chunks before the stream is terminated:

For requests with reasoning_effort set, the idle timeout uses the full request deadline timeout to accommodate long reasoning phases with no output.

Timeout Error Responses

For non-streaming requests, timeout returns HTTP 408. For streaming requests, an SSE error event is sent:

max_tokens and the Context Window

max_tokens (or max_completion_tokens) must be a positive integer; 0 or a negative value returns 400 with param: "max_tokens". There is no fixed upper limit -- the real ceiling is the endpoint's deployed context window minus your input. Your value is sent to the model verbatim: Erebine does not lower it behind your back.

When the requested output does not fit alongside the input inside the model's context window, the request returns 400 context_length_exceeded with the context length and the output budget actually available. Lower max_tokens, shorten the input, or route to an endpoint with a larger context window.

Opt In to Clamping

Clients that hardcode a large default (e.g. max_tokens: 32000) can ask Erebine to fit the value to the remaining context instead of failing:

With the clamp on, the router estimates the input token count from message length and lowers max_tokens to what is left, at minimum 1 token. max_completion_tokens takes precedence over max_tokens when both are set, and the clamp replaces whichever is in effect. The estimate is deliberately conservative, so a clamped ceiling can land below the true available budget. See Passthrough and Augmentation.

Models

List and describe the models available through your Erebine endpoint.

List Models

GET /proj_ABC123/v1/models

Lists the currently available models and their metadata.

Retrieve Model

GET /proj_ABC123/v1/models/{model}

Retrieves a model instance, providing information about the model.

Reranking and Scoring

Reranking endpoints (/v1/rerank, /v1/score) are documented on the Reranking page. The /v1/score endpoint scores a query against one or more candidate documents using the endpoint's configured reranker model.

Endpoints

List inference endpoints configured for your project.

List Endpoints

GET /proj_ABC123/v1/endpoints

Returns all non-deleted endpoints for your project, including those in provisioning, suspended, or error states.