docs
// Documentation

Documentation

An OpenAI-compatible inference surface, fully documented. Existing OpenAI SDKs and HTTP clients target it after a base-URL and key swap.

Overview

Erebine is a multi-tenant inference platform that serves open-source AI models behind an OpenAI-compatible API. Change your base URL and API key to start issuing requests against Erebine-hosted endpoints or against your own Erebine Inference Microservices (EIMs).

Key Features

  • Drop-in OpenAI SDKs - Point the official Python or Node.js client at the Erebine base URL; no fork, no shim.
  • Erebine Inference Microservice (EIM) - Self-host the same EIMs Erebine runs, behind your own perimeter, registered into the router.
  • Custom Model Upload - Push a HuggingFace directory or archive; the router preflights GPU fit and serves it under your project.
  • Streaming Support - SSE token stream with vendor-prefixed `x_*` events for reasoning, tool calls, and metadata.
  • Native KV Cache Offload - vLLM CPU KV offload plus prefix caching, on by default, to cut time-to-first-token on repeated prompts.

Ready to issue a request? Head to the Quickstart for a working curl call, or read Error Handling first if you maintain a strict SDK integration.

Getting Started

Account to first response in five minutes. Authenticate, point your client at an Erebine base URL, send a request.

API Reference

Every documented endpoint, request shape, and SSE event the router accepts and emits. Identical to OpenAI where the spec is identical; vendor-prefixed where it diverges.

Chat Completions

POST /v1/chat/completions with full tool-call, streaming, and reasoning-content support.

Responses API

POST /v1/responses for higher-level agentic flows with server-managed turns and built-in tools.

Streaming API

SSE frame format, the response.* and x_* event family, and SDK-side handling notes.

Tool Calling

Tool schemas, parallel calls, and how function results round-trip back through the model.

MCP Integration

Attach external Model Context Protocol servers; their tools surface as native tool calls.

Server-Side Tools

Web search, code interpreter, and file-search hosted by the router, no client wiring.

Web Search

Built-in web fetch with citations; usage is metered separately and surfaced in x_chat.metadata.

Image Search Sources

Openverse, Wikimedia Commons, Pexels, Pixabay, and Unsplash: the licenses each returns, the credit it needs, and whether a copy may be hosted.

Embeddings

POST /v1/embeddings with batched input, base64 output, and per-model dimension control.

Reranking & Scoring

Score query-document pairs or rerank a candidate list against a query, batch-friendly.

Images

POST /v1/images/generations and /v1/images/edits on an image generation endpoint; b64_json or signed url output, per-model sizes and limits.

Audio

POST /v1/audio/transcriptions and /translations; speech to text in json, text, srt, vtt, or verbose_json. POST /v1/audio/speech for text to speech. Audio input and spoken replies on /v1/chat/completions.

System 1 Decisions

POST /v1/systemone: choice, score, and noul answers about a state in one forward pass, on TypeSafe's Jev protocol.

Conversations

Server-stored multi-turn threads; resume by id with full reasoning and tool-call history.

Files API

Upload, list, and reference files by id from chat completions, responses, and batch.

Batch API

Submit a JSONL of requests for asynchronous processing at lower per-token cost.

Uploads API

Resumable multipart upload for files larger than the synchronous POST cap.

Stored Completions

List, retrieve, and export completions retained against a project for audit and replay.

Direct Model Dispatch

One base URL for every model in the project; the model travels in the payload, not the path.

Error Handling

OpenAI-shaped error envelopes, retry-after semantics, and which 5xx codes are safe to retry.

Features

Capabilities that extend the OpenAI surface: memory, document workspace, prefix caching, service tiers, SLO targets, and a workspace graph the router uses to route.

Model Management

Upload, share, version, and discover models.

Guides

Field-tested patterns, integrations, and the advanced flags the router exposes.

Usage Guides

Streaming patterns, rate-limit backoff, retry envelopes, and idempotency-key wiring.

SDK & Integrations

Per-SDK base-URL config: openai-python, openai-node, LangChain, LlamaIndex, opencode, Cursor.

Erebine Desktop Code

A coding agent that works in a folder on your Mac, on your project's models. Every edit and command is shown as it runs.

Erebine for iOS

Your chats, what needs you and your scheduled tasks on iPhone and iPad, with notifications for approvals and finished runs.

Claude Code

Add the Erebine MCP server to Claude Code with a project-scoped .mcp.json and a CLAUDE.md rules companion.

Claude Desktop

Paste the Erebine MCP server into Claude Desktop's config; workspace tools without a project file.

Cursor

Drop .cursor/mcp.json and an always-on rules file so Cursor calls the Erebine tools instead of asking.

OpenCode Integration

Drive Erebine endpoints from the OpenCode CLI with one-shot config import.

VS Code

Run Erebine models as native VS Code chat models via a ready-to-run Language Model Chat Provider.

Hermes Agent

Point the Hermes agent at Erebine inference and workspace tools with one user-level YAML merge.

pi

Drop a .pi/ folder that registers the workspace provider, bridges every Erebine tool and prompt, and appends the rules to pi's system prompt.

Cline

Merge an MCP server entry and a custom provider into Cline's shared settings, and drop the Erebine rules into .cline/rules/.

omp

Drop the MCP server, a model role and the Erebine rules into .omp/, and register the workspace's models in ~/.omp/agent/models.yml.

Platform

Operator surfaces around the inference surface: teams, auth, storage, billing, status, and the webhooks that tie them to your own systems.

Infrastructure

Choose between Erebine-hosted inference and self-hosted EIMs you run on your own GPUs.

EIM Guides

End-to-end walkthroughs for operating self-hosted EIMs.

Execution Management (EEM)

Long-running agent workflows that the router schedules, approves, retries, and reports on. Same authentication, same observability, same envelopes as the inference surface.

EEM Guides

Task-oriented walkthroughs for building on EEM.

Tools

Same surface the dashboard speaks, scripted. The CLI carries chat, responses, models, embeddings, rerank, batches, files, conversations, webhooks, keys, agents, slos, uploads, config, platform ops, exec, templates, approvals, and learnings.

// Most used

Most-used subcommands

All 19 subcommand pages are listed in the sidebar under Tools.