docs
// Model Management

Model Management

List, update, share, revalidate, and delete every custom model in your project from one REST surface. Mounted on the inference route group, gated by the same project API key you already use for chat completions.

Note: Model Management API endpoints use the path /proj_ABC123/v1/models. Mutations (PATCH, DELETE, POST share/unshare/revalidate/versions) run on the same inference route group as /v1/chat/completions, gated by standard project API-key authentication. There is no separate "Model Management" scope; any valid project API key is accepted, except that a PATCH changing an engine setting needs the management scope. See the Endpoint scope table below.

The Model Management API allows you to:

  • List all models in your project
  • Retrieve details for a specific model
  • Update every model setting: listing fields, capability tags, reasoning effort, engine fields, dimensions, and Hugging Face config overrides
  • Read a model's Hugging Face config overrides and preview an override set without storing it
  • Set a model's speculative decoding: method, token count, and a draft model from the catalog
  • Delete models you no longer need
  • Trigger model revalidation to re-extract metadata
  • Share a model to the public catalog or unshare it
  • List, create, promote, and roll back model versions
  • View catalog information for shared models

For uploading models, see the Model Upload guide. For versioning details, see Model Versioning.

Base URL

Every endpoint below is mounted under one project-scoped base URL. Substitute your project external id and your API key once; the remaining sections show only the path and the body.

Base URL
https://api.erebine.ai/proj_ABC123/v1
Authorization header
Authorization: Bearer ere_myproject_your_api_key

The external id has the shape ws_<hex> for workspaces and is visible in the project header. The API key format is ere_<project>_<secret> and is issued from the API Keys page.

List Project Models

GET/proj_ABC123/v1/models

Lists all models uploaded to your project. The default response is the OpenAI-compatible list envelope. Pass ?extended=true to receive Erebine-specific metadata fields (format, size, status, architecture, version, sharing, and more) along with pagination counters.

limit and offset are Erebine extensions; OpenAI's models listing defines neither. Omit limit and the standard envelope returns the whole inventory unpaginated, which is what an SDK enumerating models expects. An explicit limit is bounded to 100; a larger value is served as 100. The ?extended=true envelope defaults to 100 rows and reports the truncation on has_more and total. On THIS listing a negative limit or offset answers 400 invalid_request with param naming the one at fault, rather than being clamped silently -- see Error Responses. The inference-facing listings (an endpoint-scoped base, _direct and _semantic/router) do not refuse a negative bound; they ignore it and serve the full visible inventory.

Add catalog=public to the ?extended=true request to list the public model catalog instead of your project's models, paged the same way. Each row carries owned; a model another project published shows no storage location and no speculative decoding setting. The standard envelope ignores catalog.

curl
curl https://api.erebine.ai/proj_ABC123/v1/models \ -H "Authorization: Bearer ere_myproject_your_api_key"
Python
import requests headers = {"Authorization": "Bearer ere_myproject_your_api_key"} response = requests.get( "https://api.erebine.ai/proj_ABC123/v1/models", headers=headers ) for model in response.json()["data"]: print(f"{model['id']} ({model.get('name', '')}) owned_by={model['owned_by']}")
Node.js
const response = await fetch( "https://api.erebine.ai/proj_ABC123/v1/models", { headers: { "Authorization": "Bearer ere_myproject_your_api_key" } } ); const payload = await response.json(); for (const model of payload.data) { console.log(`${model.id} (${model.name ?? ""}) owned_by=${model.owned_by}`); }

Response (standard)

{ "object": "list", "data": [ { "id": "00000000-1111-0000-1111-000000000000", "object": "model", "created": 1706123456, "owned_by": "my-project", "name": "my-custom-model" } ] }

Response (extended)

Request with ?extended=true. Each model object carries every key of the extended read, an unset one as an explicit null; the example below shows a subset, and Get Model Details shows the full object.

{ "object": "list", "data": [ { "id": "00000000-1111-0000-1111-000000000000", "object": "model", "created": 1706123456, "owned_by": "my-project", "name": "my-custom-model", "format": "safetensors", "size_bytes": 4294967296, "size_formatted": "4.00 GB", "status": "ready", "version": "1.0.0", "is_latest": true, "architecture": "LlamaForCausalLM", "quantization": "awq", "context_length": 32768, "parameter_count": 7000000000, "workload_type": "chat", "is_shared": false } ], "has_more": false, "total": 1 }

Get Model Details

GET/proj_ABC123/v1/models/{modelId}

Retrieves detailed information about a specific model. Pass ?extended=true to include additional metadata such as format, size, architecture, and catalog status.

curl
curl https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000?extended=true \ -H "Authorization: Bearer ere_myproject_your_api_key"
Python
import requests MODEL_ID = "00000000-1111-0000-1111-000000000000" headers = {"Authorization": "Bearer ere_myproject_your_api_key"} response = requests.get( f"https://api.erebine.ai/proj_ABC123/v1/models/{MODEL_ID}", params={"extended": "true"}, headers=headers ) model = response.json() print(f"{model['name']} - {model['status']}")
Node.js
const MODEL_ID = "00000000-1111-0000-1111-000000000000"; const response = await fetch( `https://api.erebine.ai/proj_ABC123/v1/models/${MODEL_ID}?extended=true`, { headers: { "Authorization": "Bearer ere_myproject_your_api_key" } } ); const model = await response.json(); console.log(`${model.name} - ${model.status}`);

Response (standard)

{ "id": "00000000-1111-0000-1111-000000000000", "object": "model", "created": 1706123456, "owned_by": "my-project", "name": "my-custom-model" }

Response (extended)

Every key is present; a field with no value is an explicit null. speculative_decoding is null for a model your project does not own.

served_context_length is read-only: the context window the model's engine last started with, null until one has. An engine that cannot fit context_length on its host starts with a smaller window. A request may use no more than the smaller of the two, so this is the value to check when requests are refused below the context_length you set.

engine is read-only: the engine that serves the model, vllm, or laya for a System 1 model. Upload sets it from the model's files, and Update Model Metadata takes no such key. An endpoint binds the model only to Erebine Inference Microservices (EIMs) running that engine.

status_reason is read-only: why the model is in a failed status, error (upload completion refused the model) or validation_failed. It is null for every other status.

{ "id": "00000000-1111-0000-1111-000000000000", "object": "model", "created": 1706123456, "owned_by": "my-project", "name": "my-custom-model", "format": "safetensors", "size_bytes": 4294967296, "size_formatted": "4.00 GB", "status": "ready", "status_reason": null, "version": "1.0.0", "is_latest": true, "description": "A fine-tuned LLM for code generation", "architecture": "LlamaForCausalLM", "quantization": "awq", "context_length": 32768, "served_context_length": 32768, "parameter_count": 7000000000, "workload_type": "chat", "engine": "vllm", "is_shared": false, "capabilities": ["chat", "vision"], "tool_call_parser": "auto", "reasoning_parser": "auto", "scheduling_policy": "priority", "kv_cache_dtype": "auto", "expert_parallel": null, "enable_eplb": null, "all2all_backend": null, "gpu_memory_utilization": 0.9, "max_num_seqs": 128, "max_num_batched_tokens": null, "long_prefill_token_threshold": null, "enforce_eager": null, "prefix_caching": null, "disable_async_scheduling": null, "disable_custom_all_reduce": null, "engine_configs": null, "tool_narrowing": "default", "steering_role": "default", "family_reasoning_efforts": ["low", "medium", "high"], "reasoning_efforts_source": "profile", "is_pre_quantized": true, "pre_quantization_method": "awq", "pre_quantization_bits": 4 }

Update Model Metadata

PATCH/proj_ABC123/v1/models/{modelId}

Updates metadata on a model your project owns. Every key in the request body is optional and three-state: leave a key out to keep the stored value, send null to clear it, or send a value to set it. The whole body is checked and stored in one transaction: a key the server refuses returns an error whose param names it, and nothing of the body is stored.

An engine setting is a key the engine launches with: capabilities, quantization, context_length, the two parsers, trust_remote_code, scheduling_policy, kv_cache_dtype, the engine tuning keys, parameter_count, num_layers, hidden_size, torch_dtype, the expert counts, the pre-quantization keys, speculative_decoding and hf_overrides. A body that changes one needs an API key with the management scope, and every endpoint serving the model learns the stored value at once. A change to quantization, scheduling_policy, kv_cache_dtype, an engine tuning key, speculative_decoding or hf_overrides, or to capabilities that changes the model's input modalities, restarts those engines with it; an endpoint that sets its own kv_cache_dtype keeps that value and does not restart for a model kv_cache_dtype change. The other engine settings apply at each engine's next launch. A body that re-sends an engine setting's stored value changes nothing and needs no extra scope, so a client may send back what it read.

A config-derived value you set (the description, license, Hugging Face id, architecture, quantization, context length, dimensions, dtype, expert counts and pre-quantization keys) is kept: Revalidate and a metadata refresh re-read the model's files only for the fields you have not edited. Clearing such a field (null) hands it back to the files: the next revalidation or refresh fills it again.

Request Body

Parameter Type Description
descriptionoptional string or null Markdown description of the model. null or "" clears it.
licenseoptional string or null License identifier (e.g., "Apache-2.0"). null or "" clears it.
hugging_face_idoptional string or null Upstream Hugging Face repository id (e.g. meta-llama/Llama-3.1-8B-Instruct, or a single-segment id such as gpt2). null or "" clears it. A value that cannot be a repository id returns HTTP 400 with "Invalid Hugging Face repository id: <value>". Detected from config.json's _name_or_path at import when the checkpoint carries one; never guessed from a file or directory name.
workload_typeoptional string Comma-separated workload tags: chat, code, reasoning, embedding, multilingual. At least one; it cannot be cleared.
capabilitiesoptional array of strings The whole capability tag list: chat, vision, video_input, audio_input, transcription, speech, realtime, forced_tool_choice. Requests are admitted per tag, and an endpoint's task mode is checked against them. [] clears the tags. structured_output is set only by the capability probe and is kept whatever you send. An unknown tag returns HTTP 400.
reasoning_effortsoptional array of strings or null The reasoning_effort values the model accepts: [] for a model that does not reason, or a list drawn from none, minimal, low, medium, high, xhigh, max. A list you send is stored as yours (reasoning_efforts_source operator). null resets the model to its detected list; the response carries that list and its source. A value the model cannot honour returns HTTP 400, and so does a list on a model served without a reasoning parser. A model whose capability tags do not include chat, such as a speech, image generation or transcription model, takes only null. See Reasoning Effort.
architectureoptional string or null Model architecture (e.g., "LlamaForCausalLM"). null or "" clears it.
quantizationoptional string or null Runtime quantization method: native, awq, bitsandbytes, bitblas, gguf, gptq, ipex, int4, int8, fp8, modelopt, quark, torchao, or compressed-tensors. null or "" resets it to native. Any other value returns HTTP 400 with "Invalid quantization method: <value>".
context_lengthoptional integer Maximum context length in tokens, at least 128; it is the engine's --max-model-len and cannot be cleared. An engine that cannot fit it starts with a smaller window, reported as served_context_length in the extended read; your value is kept and asked for again at the next engine start.
tool_call_parseroptional string or null The engine's tool-call parser. auto (also null or "") detects it from the architecture.
reasoning_parseroptional string or null The engine's reasoning parser. auto (also null or "") detects it from the architecture.
trust_remote_codeoptional boolean Whether the engine may execute code shipped in the model bundle. A change needs the project owner; it cannot be cleared.
scheduling_policyoptional string or null The engine's --scheduling-policy: priority (the default) serves realtime and interactive requests ahead of batch and background work; fcfs serves every request in arrival order. null or "" resets it to priority. Any other value returns HTTP 400. A System 1 model takes only priority.
kv_cache_dtypeoptional string or null The engine's --kv-cache-dtype. auto (the default) passes no flag, so the engine chooses: a checkpoint whose quantization config declares an fp8 or nvfp4 KV cache (a ModelOpt kv_cache_quant_algo or kv_cache_scheme, or an llm-compressor kv_cache_scheme, which MLA models such as DeepSeek V3 do not apply) runs that, and any other model keeps its KV cache in the model dtype, bfloat16 or float16. Fp8 weights alone do not make the KV cache fp8. Any other value is passed as chosen: bfloat16, float16, fp8, fp8_ds_mla, fp8_e4m3, fp8_e5m2, fp8_inc, fp8_per_token_head, int4_per_token_head, int8_per_token_head, nvfp4, nvfp4_4over6, turboquant_3bit_nc, turboquant_4bit_nc, turboquant_k3v4_nc, or turboquant_k8v4. null or "" resets it to auto; any other value returns HTTP 400. An endpoint's own kv_cache_dtype overrides it. A node whose engine does not accept the value refuses to start the endpoint and says which value it refused. A System 1 model takes only auto.
expert_parallel, enable_eplb, all2all_backend, gpu_memory_utilization, max_num_seqs, max_num_batched_tokens, long_prefill_token_threshold, enforce_eager, prefix_caching, disable_async_scheduling, disable_custom_all_reduceoptional string, number, boolean or null The engine tuning: vLLM launch settings for every engine serving the model. null is auto, the EIM's own choice, for each. A value outside a key's vocabulary or range returns HTTP 400. A System 1 model takes only null. See Engine Tuning.
engine_configsoptional object or null vLLM JSON configs, one object per flag: attention_config, compilation_config, speculative_config, reasoning_config. A flag's object replaces its stored object, a flag's null clears it, and a flag left out keeps its object; null for the whole key clears every flag. An unknown flag, a value that is not an object, or a key no supported vLLM release accepts returns HTTP 400 naming it. A System 1 model takes only null. See vLLM JSON configs.
image_generationoptional object or null An image generation model's own settings: the limits and defaults every image request to it uses. The object replaces the stored one whole; null or {} clears it, so the model's profile applies unchanged. Refused with HTTP 400 on a model that is not an image generation model, and for a value the profile cannot serve. See Image Generation.
speechoptional object or null A speech model's voice settings, an object with disabled_voices, default_voice and aliases: the voices it does not offer, its default voice and the OpenAI voice names mapped by hand. The object replaces the stored one whole; null or {} clears it, so the model's own choices apply. Refused with HTTP 400 on a model with no voices, and for a voice the model does not offer. See Speech Voices.
tool_narrowingoptional string or null Whether the model's chat turns start with a narrowed tool block: default (the platform setting), on or off. null or "" is default. Any other value returns HTTP 400. A model whose capability tags do not include chat takes only default. See Chat Options.
steering_roleoptional string or null How the router's mid-turn corrections reach the model: default (system instructions) or user_note (a marked platform note after the tool results). null or "" is default. user_note returns HTTP 400, with the reason, on a model whose chat template does not render a user turn after tool results as written. See Chat Options.
parameter_countoptional integer or null Total number of model parameters; sizes the VRAM estimate. null or 0 clears it.
num_layersoptional integer or null Number of transformer layers. null or 0 clears it.
hidden_sizeoptional integer or null Hidden dimension size. null or 0 clears it.
vocab_sizeoptional integer or null Vocabulary size. null or 0 clears it.
torch_dtypeoptional string or null Native PyTorch dtype: float16, bfloat16, or float32. null clears it; any other value returns HTTP 400.
num_expertsoptional integer or null Total number of mixture-of-experts experts. null or 0 clears it.
num_experts_per_tokoptional integer or null Experts activated per token. null or 0 clears it.
is_pre_quantizedoptional boolean Whether the weights are stored in quantized form; it cannot be cleared.
pre_quantization_methodoptional string or null compressed-tensors, gptq, awq, or fp8. null clears it; any other value returns HTTP 400.
pre_quantization_bitsoptional integer or null 4 or 8. null clears it; any other value returns HTTP 400.
speculative_decodingoptional object or null The model's speculative decoding setting. The keys you send are merged onto the stored setting and the result is checked as a whole; each key takes a value, or null to clear it. Send null for the whole object to reset the setting to off with nothing chosen. A setting that breaks a rule returns HTTP 400 naming the rule. A model whose capability tags do not include chat takes only null or an off. See Speculative Decoding for the keys.
hf_overridesoptional array of objects or null The model's Hugging Face config override rows, replaced as a whole: each {"key_path", "value_type", "value"}, where value_type is string, number or boolean. null or [] removes every row. A set that does not assemble returns HTTP 422 naming the row. See HF Config Overrides.
curl
curl -X PATCH https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000 \ -H "Authorization: Bearer ere_myproject_your_api_key" \ -H "Content-Type: application/json" \ -d '{ "description": "Fine-tuned for code generation tasks", "architecture": "LlamaForCausalLM", "context_length": 32768 }'
Python
import requests MODEL_ID = "00000000-1111-0000-1111-000000000000" headers = { "Authorization": "Bearer ere_myproject_your_api_key", "Content-Type": "application/json" } response = requests.patch( f"https://api.erebine.ai/proj_ABC123/v1/models/{MODEL_ID}", headers=headers, json={ "description": "Fine-tuned for code generation tasks", "architecture": "LlamaForCausalLM", "context_length": 32768 } ) model = response.json() print(f"Updated: {model['name']}")
Node.js
const MODEL_ID = "00000000-1111-0000-1111-000000000000"; const response = await fetch( `https://api.erebine.ai/proj_ABC123/v1/models/${MODEL_ID}`, { method: "PATCH", headers: { "Authorization": "Bearer ere_myproject_your_api_key", "Content-Type": "application/json" }, body: JSON.stringify({ description: "Fine-tuned for code generation tasks", architecture: "LlamaForCausalLM", context_length: 32768 }) } ); const model = await response.json(); console.log(`Updated: ${model.name}`);

Response

The extended model as stored, every key present: a field with no value is an explicit null, so a client re-bases on what was stored rather than on what it sent.

{ "id": "00000000-1111-0000-1111-000000000000", "object": "model", "created": 1706123456, "owned_by": "my-project", "name": "my-custom-model", "format": "safetensors", "size_bytes": 4294967296, "size_formatted": "4.00 GB", "status": "ready", "version": "1.0.0", "is_latest": true, "description": "Fine-tuned for code generation tasks", "architecture": "LlamaForCausalLM", "quantization": "native", "context_length": 32768, "parameter_count": null, "workload_type": "chat", "is_shared": false, "license": null, "capabilities": ["chat"], "tool_call_parser": "auto", "reasoning_parser": "auto" }

HF Config Overrides

Hugging Face config overrides change values of a model's config.json at engine launch, for example a RoPE or YaRN scaling factor. Each row sets one dotted key path; an all-digit segment indexes an array. Set them with hf_overrides on Update Model Metadata. They do not combine with an enabled speculative decoding setting. Both read routes answer only for a model your project owns.

GET/proj_ABC123/v1/models/{modelId}/hf-overrides

Returns the stored rows in key-path order.

curl
curl https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000/hf-overrides \ -H "Authorization: Bearer ere_myproject_your_api_key"

Response

{ "rows": [ {"key_path": "rope_scaling.factor", "value_type": "number", "value": "4.0"}, {"key_path": "rope_scaling.rope_type", "value_type": "string", "value": "yarn"} ] }

POST/proj_ABC123/v1/models/{modelId}/hf-overrides/preview

Assembles a proposed row set into the document the engine receives, and stores nothing. The body has the same shape as the list response. A set that does not assemble, or an empty set, returns HTTP 422 naming the problem.

curl
curl -X POST https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000/hf-overrides/preview \ -H "Authorization: Bearer ere_myproject_your_api_key" \ -H "Content-Type: application/json" \ -d '{"rows": [{"key_path": "rope_scaling.factor", "value_type": "number", "value": "4.0"}]}'

Response

{ "json": "{\"rope_scaling\":{\"factor\":4.0}}" }

Reasoning Effort

Every model holds the list of reasoning_effort values it accepts. The list governs reasoning_effort on chat completions, reasoning.effort on the Responses API, and the reasoning object's effort, enabled and max_tokens. It is read on every path that takes them: endpoint-scoped routes, _direct, and the semantic routes.

Erebine detects the list when a model is imported, and again when its chat template, architecture or reasoning parser changes. A model served without a reasoning parser is detected as one that does not reason. A reasoning model gets every value its family accepts: the levels its chat template checks for when it checks, otherwise the levels of its model family, together with the OpenAI values that map onto them. You can replace the detected list with your own. A list you set is never changed by detection; Reset to detected returns to it.

Setting reasoning_efforts Behaviour
Does not reason [] Every reasoning control is refused with HTTP 400 unsupported_parameter, as OpenAI refuses reasoning_effort on a non-reasoning model. reasoning.exclude and include_reasoning are accepted: they only shape the response, and there is no reasoning to leave out. The model's provider listing does not advertise reasoning.
Only these values a list Only the listed values are accepted; any other returns HTTP 400 unsupported_value naming the list. reasoning.enabled: false counts as none, and reasoning.enabled: true alone counts as medium.

There is no third, default setting. A request to update a model that sends reasoning_efforts: null, as older clients did to choose the family default, resets the model to its detected list.

Where the list came from

A model's reasoning_efforts_source says where its list came from, and the dashboard shows the same in words under the setting.

reasoning_efforts_source Meaning
operator You set the list. Detection and the reasoning probe leave it alone.
template Detected. The model's chat template checks reasoning_effort against a list of levels, and those are the levels the engine takes.
profile Detected from the model family. When neither the chat template nor the family names levels, each listed value is sent to the engine as it arrived.
probe Detected, then narrowed: the reasoning probe found a value the engine refuses and removed it.

What the engine is sent

An accepted value is expressed in the model family's own terms before it reaches the engine. minimal is sent as the family's lowest level and max as its highest; a family whose levels are low, medium and xhigh is sent xhigh for high; none turns thinking off with the family's own switch; and a level that needs a large context window is sent as the next level down on an endpoint that serves less. The response reports a value that was sent differently in x_adjusted_reasoning_effort. This translation belongs to the model family and is not a setting. The model's Request settings section lists it per accepted value, and erectl models show prints the same lines.

The reasoning probe

The capability probe also checks reasoning on a deployed model: it sends a few short requests, one per distinct level, and records which values the engine accepts and whether none returns a reply without reasoning. A value the engine refuses is removed from a detected list. On a list you set, nothing is removed: the dashboard marks the value and warns beside it, and you decide. A probe that cannot reach the model changes nothing. The probe does not measure whether one level thinks harder than another.

On a semantic route the router picks the model, so a request cannot know which one answers. There a model that does not reason has the reasoning controls dropped, and a listed model takes the nearest listed value, instead of the request being refused. A request that names the model or endpoint gets the refusal.

Set it

In the dashboard, open the model, go to its Request settings section, pick Does not reason or Only these values in the Reasoning effort row, tick the values, and save the row. The row shows where the current list came from and offers Reset to detected when the list is yours. The row appears for a model tagged chat; a speech, image generation or transcription model has none. Through the API, send reasoning_efforts to Update Model Metadata. From the CLI:

erectl
erectl models update 00000000-1111-0000-1111-000000000000 --reasoning-efforts unsupported erectl models update 00000000-1111-0000-1111-000000000000 --reasoning-efforts low,medium,high erectl models update 00000000-1111-0000-1111-000000000000 --reasoning-efforts detected

--reasoning-efforts default, the keyword older scripts used, still works and means detected.

Refusal on a model that does not reason

{ "error": { "message": "Unsupported parameter: 'reasoning_effort' is not supported with this model.", "type": "invalid_request_error", "param": "reasoning_effort", "code": "unsupported_parameter" } }

Chat Options

Two settings decide how chat turns on a model are built. Both apply to the next chat turn after a save, restart nothing, and apply to every project that uses the model.

Tool narrowing

A narrowed chat turn starts with a core of tools (memory, workspace search, web search and fetch, the clock, asking you) and a catalog the model loads the other tool groups from when the turn needs them. That cuts the tool definitions sent with each model call by about four fifths, so turns start faster and cost fewer input tokens. It relies on the model loading a group when it needs one; a model that does not answers without the tool. Pick it for a model that uses the catalog well.

Value Chat turns on the model
default Follow the platform setting.
on Narrowed.
off Never narrowed: every tool is offered except the plan and mockup tools a turn cannot use yet, which the catalog holds.

Only chats in Erebine's own apps are narrowed. A request made with an API key gets every tool it names, whatever the model's setting.

Mid-turn steering

Between tool calls the router sometimes corrects the model: asking for the answer once the sources are in, for a tool call with the arguments it left out, or for a clarifying question first. By default a correction is a system instruction. The engine then reads every tool definition again on that call, because the system text in front of them changed. With user_note the correction is a marked platform note after the tool results, and the engine serves everything before it from its prompt cache. How closely a model follows a note can differ from how it follows a system instruction.

Mid-turn steering also applies to Responses API requests on the model that send corrections: a request with the web search tool, and any request that turns on corrective retries (X-Erebine-Augment-Corrective-Retries: on). The note is never stored: the conversation, the response and an export keep the turn as it was. user_note is offered only on a model whose chat template renders a user turn after tool results as written; on any other model the setting is refused with the reason, and the dashboard says why beside the row.

Set them

In the dashboard, open the model and go to its Request settings section: Tool narrowing and Mid-turn steering each save on their own. Through the API, send tool_narrowing or steering_role to Update Model Metadata; Get Model Details returns both. From the CLI:

erectl
erectl models update 00000000-1111-0000-1111-000000000000 --tool-narrowing on erectl models update 00000000-1111-0000-1111-000000000000 --steering-role user_note erectl models update 00000000-1111-0000-1111-000000000000 --tool-narrowing default --steering-role default

Speculative Decoding

Speculative decoding drafts several tokens ahead and has the model verify them in one forward pass, so a response streams faster without changing what the model samples. It is a setting on the model. Every endpoint that serves the model uses it, and every EIM that runs one of those endpoints applies it when it starts the engine: the EIM downloads the draft model, verifies it like any model, and passes it to the engine. Nothing is set on the EIM and no files are placed by hand.

The setting

Key Type Description
enabled boolean The model speculates by default on every endpoint. Requires a method. Defaults to false.
method string or null One of the methods below.
num_speculative_tokens integer or null Tokens drafted per step, 1 to 32. Unset uses the method's default. More tokens pay off only while most of them are accepted.
draft_model_id string or null The id of the model that drafts. Required by the methods that run a draft and refused by the others.
ngram_fallback boolean mtp only: speculate with ngram when the model's checkpoint ships no multi-token-prediction layers, instead of not speculating. Defaults to false.
temperature, top_p, top_k number or null Sampling defaults while the engine speculates: temperature 0 to 2, top_p above 0 and at most 1, top_k 1 or more. See Sampling defaults.

Methods

Method Draft model What drafts
mtp Refused The model's own multi-token-prediction layers. The checkpoint must ship them; ngram_fallback covers one that does not.
ngram Refused Prompt lookup: continues an n-gram already in the context. Pays off on output that repeats its input, such as code edits and extraction.
eagle, eagle3 Required An EAGLE or EAGLE-3 head trained for this model.
dflash Required A DFlash drafter trained for this model.
draft_model Required A smaller model that shares this model's tokenizer.
medusa, mlp_speculator Required Medusa or MLP speculator heads trained for this model. vLLM 0.33.0 removed both methods: an EIM on that release or a later one refuses to launch the model with either, and an EIM on an older engine still runs them.

The draft model

A draft is a model in your catalog: one of your project's models, or a model in the shared catalog. Upload it like any other model; its name does not matter. It cannot be the model itself or a preseeded model, it cannot have a draft of its own, and a model that is another model's draft cannot take one. A draft still validating can be chosen ahead of time; it is used once it is ready.

Saving the setting, on the model page or through the API, restarts the engines on endpoints that serve the model with the new setting. When the method needs a draft, the engine waits up to five minutes for a draft still downloading, then starts without speculative decoding. A draft no longer named is no longer held for the model and can be evicted to free space. A model that is another model's draft cannot be deleted: Delete Model returns 409 naming the models in your project that use it and counting the rest. A shared model that a model in another project uses as its draft cannot leave the catalog: Unshare Model returns 409 counting those models. Deleting the endpoints that serve it does not take it out of the catalog either, and removing the upload that created it leaves the model in place.

With draft_model, the draft must have the same vocabulary size as the model when both are known; the engine refuses the pair otherwise, so the setting is refused with HTTP 400 instead. An engine older than the release that introduced the chosen method starts without speculative decoding rather than failing to start.

When an endpoint speculates

  1. An endpoint's own Speculative decoding setting wins: off never speculates; on speculates with this setting's method, or with the model's own multi-token-prediction layers when no method is chosen.
  2. On auto, the endpoint follows the model: enabled speculates. A model not enabled speculates only on an endpoint whose workload profile is latency.
  3. The operator of a self-hosted EIM can override the setting for every model on that EIM; see EIM Advanced Configuration.

Sampling defaults

temperature, top_p and top_k fill in only the parameters a request leaves unset, and only while the engine speculates from this setting. A request's own value always wins.

They exist for thinking budgets. With speculative decoding on, vLLM 0.30.0 through 0.32 corrupts a request that reaches its thinking budget when the request samples with top_p below 1 and no top_k (vllm-project/vllm#58231, fixed in vLLM 0.33.0). On an EIM whose engine is one of those releases, a budget is therefore accepted only from a request that is greedy (temperature 0), sends top_p 1.0, or carries a top_k; any other request that sends reasoning.max_tokens gets HTTP 400 naming these ways out, and a budget derived from reasoning_effort is not sent. A top_k default makes every request that sends no top_k safe; top_p 1.0 or temperature 0 covers only requests that leave those unset.

HF config overrides

Speculative decoding and HF config overrides do not combine: the engine applies overrides to the model and not to its draft (vllm-project/vllm#37435), so a RoPE or YaRN override would have the model verify at one context geometry while the draft proposes at another. Enabling speculative decoding on a model with overrides is refused, and so is saving overrides on a model that speculates. When an endpoint turns speculation on for a model that has overrides, the engine launches without the overrides.

Set it

In the dashboard, open the model and use its Speculative decoding section, which a model tagged chat has: pick the method and its draft and save that row, set the token count and save it, then turn on Enabled, which saves as it flips. Through the API, send speculative_decoding to Update Model Metadata; an extended read of a model you own returns the setting under the same key. From the CLI:

erectl
erectl models update 00000000-1111-0000-1111-000000000000 --speculative on --speculative-method dflash \ --speculative-tokens 7 --draft-model 00000000-2222-0000-2222-000000000000 erectl models update 00000000-1111-0000-1111-000000000000 --speculative on --speculative-method mtp --ngram-fallback on erectl models update 00000000-1111-0000-1111-000000000000 --speculative-top-k 20 erectl models update 00000000-1111-0000-1111-000000000000 --speculative off erectl models update 00000000-1111-0000-1111-000000000000 --speculative-reset
curl
curl -X PATCH https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000 \ -H "Authorization: Bearer ere_myproject_your_api_key" \ -H "Content-Type: application/json" \ -d '{ "speculative_decoding": { "enabled": true, "method": "dflash", "num_speculative_tokens": 7, "draft_model_id": "00000000-2222-0000-2222-000000000000", "top_k": 20 } }'

In the extended read

"speculative_decoding": { "enabled": true, "method": "dflash", "num_speculative_tokens": 7, "draft_model_id": "00000000-2222-0000-2222-000000000000", "ngram_fallback": false, "top_k": 20 }

Refusal of a draft method with no draft

{ "error": { "message": "Method dflash runs a draft model: choose one from the catalog.", "type": "invalid_request_error", "code": "invalid_request" } }

Engine Tuning

Eleven vLLM launch settings live on the model, so every engine serving it launches the same way, whichever EIM runs it. Each is null, auto, until you set it: auto is the EIM's own choice, exactly what it launches with when the model states nothing. A change restarts the engines serving the model with the new value; it needs an API key with the management scope. A System 1 model takes only null: its engine does not run vLLM.

The settings

Key Type Description
expert_parallel string or null --enable-expert-parallel, mixture-of-experts models on more than one GPU. on forces it, off keeps it off. Auto turns it on for a mixture-of-experts model on more than one GPU. An EIM with auto-configuration off ignores it.
enable_eplb boolean or null --enable-eplb, expert parallel load balancing, with the engine's default balancing window. true turns it on; false reads as null. It applies only with expert parallel across more than one rank, tensor parallel size times data parallel size, on NVIDIA CUDA. Refused while expert_parallel is off. A quantization the engine cannot balance makes the EIM drop it and keep expert parallel. Auto leaves it off.
all2all_backend string or null --all2all-backend under expert parallel: allgather_reducescatter, flashinfer_nvlink_two_sided or flashinfer_nvlink_one_sided. A FlashInfer backend is not used on AMD ROCm. Auto picks flashinfer_nvlink_two_sided across NVLink nodes and the engine's default otherwise. An EIM with auto-configuration off ignores it.
gpu_memory_utilization number or null --gpu-memory-utilization, above 0 and at most 1. On WSL the EIM still lowers it to leave the host its headroom. Auto is the EIM's value, 0.95 unless its operator set one.
max_num_seqs integer or null --max-num-seqs, 1 or more. Auto lets the EIM size it from the GPU and the model.
max_num_batched_tokens integer or null --max-num-batched-tokens, 1 or more, and refused below max_num_seqs when both are set. A value below what the engine accepts at launch is not sent, and the engine's default budget applies. Auto lets the EIM size it.
long_prefill_token_threshold integer or null --long-prefill-token-threshold, 0 or more; 0 is no cap. A value above the context length is not sent. Auto derives it from the step budget.
enforce_eager boolean or null --enforce-eager: run without CUDA graphs. true turns it on; false reads as null. Auto uses CUDA graphs unless the EIM turns them off for a GPU that needs it.
prefix_caching string or null Automatic prefix caching: on or off. Audio models always run without it, and embedding and scoring endpoints are left to the engine. Auto keeps it on. See Prefix Caching.
disable_async_scheduling boolean or null --no-async-scheduling. true turns async scheduling off; false reads as null. A model cannot force it on. AMD ROCm engines ignore it. Auto lets the EIM and the engine decide.
disable_custom_all_reduce boolean or null --disable-custom-all-reduce: every all-reduce across GPUs uses NCCL. true turns it on; false reads as null. Auto keeps the custom kernel unless the EIM turns it off for a GPU that needs it.

Four settings only add a flag: enable_eplb, enforce_eager, disable_async_scheduling and disable_custom_all_reduce. The EIM turns CUDA graphs and the custom all-reduce kernel off on GPUs that fail with them, and a model cannot undo that.

Mixture of experts

expert_parallel, enable_eplb and all2all_backend are refused on a model whose num_experts is 1 or less, judged on the model as the request leaves it. An EIM that reads the model's own config as dense ignores them.

Who wins

Per setting, highest first:

  1. The EIM operator: an option, environment variable or vLLM argument set on that EIM. A vLLM argument wins in any spelling the engine accepts.
  2. The model: the value set here.
  3. Auto: the EIM's own choice.

A model value the operator overrode, or one the launch cannot use, is logged by the EIM with the value that won. An EIM that predates the settings launches as if every setting were auto.

vLLM JSON configs

Some vLLM settings take one JSON object per flag. engine_configs sets four of them: attention_config (--attention-config), compilation_config (--compilation-config), speculative_config (--speculative-config) and reasoning_config (--reasoning-config). Each object is merged over the object the EIM derives for that flag: a key you set wins, and a key set to null removes the EIM's value. An EIM operator's own value for the flag replaces the result on that EIM. Only the top-level keys are checked: each must be a field of the flag's vLLM class in a vLLM release the EIM images ship, and a refusal names the flag, the key and the releases. A key only an older release reads is refused with the key that replaced it named: use_fp4_indexer_cache is refused in favour of "indexer_kv_dtype": "mxfp4".

Set it

In the dashboard, open the model and use its Engine tuning section, where every choice opens on Auto and an empty number field is auto; each setting is a row that saves on its own, and the vLLM JSON configs rows take the JSON objects, one row per flag. Through the API, send the keys to Update Model Metadata; an extended read returns them, null for auto. From the CLI, see the engine tuning options:

erectl
erectl models update 00000000-1111-0000-1111-000000000000 --max-num-seqs 128 --gpu-memory-utilization 0.9 erectl models update 00000000-1111-0000-1111-000000000000 --expert-parallel on --all2all-backend flashinfer_nvlink_two_sided erectl models update 00000000-1111-0000-1111-000000000000 --max-num-seqs auto erectl models update 00000000-1111-0000-1111-000000000000 --engine-config attention_config='{"indexer_kv_dtype":"mxfp4"}'
curl
curl -X PATCH https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000 \ -H "Authorization: Bearer ere_myproject_your_api_key" \ -H "Content-Type: application/json" \ -d '{ "max_num_seqs": 128, "gpu_memory_utilization": 0.9, "prefix_caching": "off", "enforce_eager": null }'

Image Generation

Most diffusion image models the image engine serves work as uploaded or imported: their model_index.json names the pipeline, the model is tagged image_generation, and an image_generation endpoint serves it on /v1/images/generations, /v1/images/edits, the chat image tool, the Responses image_generation tool and MCP (see Images). Only an image_generation endpoint serves an image pipeline; any other task mode is refused with HTTP 400. A video or audio pipeline is not an image generation model: it gets no tag and no task mode serves it. The EIM needs vLLM-Omni, which the eim-vllm-omni-cuda and eim-vllm-omni-omix images ship.

Every image model starts from a profile, the limits its requests are checked against:

  • Curated profile: FLUX.2 klein, HiDream-O1-Image and Z-Image-Turbo, with limits read from the engine for each pipeline (table below).
  • Auto profile: any other image pipeline. One image per request; edits only for a pipeline the engine declares takes input images, as many as it declares; free sizes from 256 to 2048 pixels a side in multiples of 16, at most 4 megapixels; the default size the checkpoint's configs name, else 1024x1024; the inference steps and guidance scale the model card's front matter names, else the engine's own defaults; and a GPU memory floor of the loaded weights plus 20 percent.
Curated pipeline Images per request Edit inputs Sizes Default size Steps
FLUX.2 klein 1 to 4 up to 4 free: 256 to 2048 a side, multiples of 16, at most 4 megapixels 1024x1024 4
HiDream-O1-Image 1 none (text to image only) fixed: 2048x2048, 2304x1728, 1728x2304, 2560x1440, 1440x2560, 2496x1664, 1664x2496, 3104x1312, 1312x3104, 2304x1792, 1792x2304 2048x2048 50
Z-Image-Turbo 1 to 4 none (text to image only) free: 256 to 2048 a side, multiples of 16, at most 4 megapixels 1024x1024 9, guidance scale 0

Every image pipeline loads at BF16 and is never quantized. An endpoint for an auto-profiled model is created only when a registered EIM's engine can load the pipeline, and binds only EIMs whose engine can. An EIM whose engine reports no pipeline list loads the curated pipelines only.

The settings

Each key overrides one value of the profile; an absent key keeps it. They apply to the next request, on every surface, and restart nothing. The keys are the snake-case spellings below; the dashboard sends the same object.

Key Type Description
max_images_per_request integer The most images one request may ask for (n), 1 to 10.
max_input_images integer The most input images one edit may carry, 0 to 16. 0 serves no edits.
fixed_sizes array of strings The only sizes rendered, as WIDTHxHEIGHT, 1 to 32 of them, each side 64 to 4096 and a multiple of 8. Any other requested size snaps to the closest ratio. Refused together with the size range keys below.
min_side, max_side integer A free size's smallest and largest side in pixels, 64 to 4096, multiples of size_multiple, smallest at most largest.
size_multiple integer The multiple every side of a free size must be: 8, 16, 32 or 64.
max_pixels integer A free size's pixel ceiling, at most 16 megapixels (16 x 1024 x 1024), and at least a square of min_side.
default_size string The size size: "auto" renders, as WIDTHxHEIGHT. The size settings must allow it.
inference_steps integer The denoising steps every request runs, 1 to 200. A step-distilled model needs its own count.
guidance_scale number The classifier-free guidance scale every request runs, 0 to 20.
min_vram_per_gpu_bytes integer The memory each GPU must have before an engine is placed on it, 1 GB to 1000 GB, in bytes.
engine_parameters object Advanced image engine parameters every request sends, each only on the routes that read it: negative_prompt (at most 2000 characters), true_cfg_scale (0 to 20), seed (0 or more; the same prompt then renders the same image), flow_shift (above 0, at most 100; generations only), guidance_scale_2 (0 to 20; edits only) and strength (0 to 1; edits only).

The settings are the model's defaults, not request parameters: the Images API takes no steps or guidance from a client.

Set it

In the dashboard, open the model and use the Image generation rows of its Request settings section, which an image model shows in place of Engine tuning. Each field shows the value in effect and where it comes from, curated, auto or override; an empty field keeps the profile's value, and Reset to profile empties one. The model catalog shows the same values. Through the API, send the object to Update Model Metadata:

curl
curl -X PATCH https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000 \ -H "Authorization: Bearer ere_myproject_your_api_key" \ -H "Content-Type: application/json" \ -d '{ "image_generation": { "inference_steps": 28, "guidance_scale": 4.0, "max_images_per_request": 2, "engine_parameters": {"negative_prompt": "blurry, low quality"} } }'

Speech Voices

A speech model's voices are read from its checkpoint and confirmed by the engine that serves it; the request side is on Audio, Voices. Three settings change which voices a request reaches. They apply to the next request on /v1/audio/speech and in chat audio replies, and restart nothing. Names compare without regard to case and are stored in the model's own spelling.

Key Type Description
disabled_voices array of strings Voices of the model it does not offer. A request naming one, directly or through an OpenAI name, returns a 400 on voice. At least one voice must stay offered.
default_voice string The voice default, an empty voice and an omitted one speak in. Unset, it is the model's voice named default, else its first offered voice in name order. It must be an offered voice.
aliases object OpenAI voice names mapped to voices of the model, such as {"alloy": "serena"}. A name left out keeps the voice the model derives for it. Each key must be one of the thirteen OpenAI names and each value an offered voice.

In the dashboard, open the model and use the Speech rows of its Request settings section: a switch per voice, the default voice, a select per OpenAI name that shows the derived voice until you choose one, and a sample of each voice. Through the API, send the object to Update Model Metadata:

curl
curl -X PATCH https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000 \ -H "Authorization: Bearer ere_myproject_your_api_key" \ -H "Content-Type: application/json" \ -d '{ "speech": { "disabled_voices": ["uncle_fu"], "default_voice": "serena", "aliases": {"alloy": "serena", "echo": "ryan"} } }'

Import a voice pack

POST/proj_ABC123/v1/models/{modelId}/speech/voice-pack

Fetches a voice pack from a public Hugging Face repository into a model served by the native speech engine: the repository's voices.jsonl manifest (or voices.json) and every voice file it names under voices/. Only the voices in the model's languages are imported. The pack replaces the model's voices/ directory, everything is fetched before anything is written, and the EIMs serving the model sync the files and restart their engines. Needs the management scope. Phonon's published pack is gradium/phonon-voices.

Parameter Type Description
repositoryrequired string The repository, owner/name. A gated or private repository is refused.
revisionoptional string A branch, tag or commit: letters, digits, ., _ and -. Defaults to main.
curl
curl -X POST https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000/speech/voice-pack \ -H "Authorization: Bearer ere_myproject_your_api_key" \ -H "Content-Type: application/json" \ -d '{"repository": "gradium/phonon-voices"}'

Response

{ "id": "00000000-1111-0000-1111-000000000000", "repository": "gradium/phonon-voices", "revision": "main", "files_imported": 38, "voices": ["Zoey", "default"] }
HTTP Status Meaning
400The body, the repository, the revision or a manifest row is not valid, the model is not a speech model, or it is not served by the native speech engine. The message names the field; nothing was written.
403The API key lacks the management scope (code scope_insufficient).
409The model is preseeded and the platform holds none of its files, or another import into the model is running (code import_in_progress).
502The repository or a voice file could not be fetched (code upstream_error). Nothing was written.
503Voice pack import is not configured: the router has no model storage, and importing again does not help. Or model storage failed partway through an import: import the same pack again to finish it.

Delete Model

DELETE/proj_ABC123/v1/models/{modelId}

Permanently deletes a model from your project. This action cannot be undone.

What goes with it: The model's files are deleted from storage, or, for a model an EIM announced, every EIM holding a copy deletes it. Its catalog entry and the records that belong to it go too. This action cannot be undone.

Requirements

  • Valid project API key (the standard inference API-key authentication)
  • The model must belong to the calling project
  • No endpoint may serve it. While a live endpoint or anchor endpoint serves the model, the delete returns 409 naming your endpoints and counting the rest, and nothing is deleted. Delete those endpoints first.
  • No other model may use it as its speculative decoding draft. The delete returns 409 naming those models; change or clear their draft first. See Speculative Decoding.
curl
curl -X DELETE https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000 \ -H "Authorization: Bearer ere_myproject_your_api_key"
Python
import requests API_KEY = "ere_myproject_your_api_key" BASE_URL = "https://api.erebine.ai/proj_ABC123/v1" MODEL_ID = "00000000-1111-0000-1111-000000000000" response = requests.delete( f"{BASE_URL}/models/{MODEL_ID}", headers={"Authorization": f"Bearer {API_KEY}"} ) if response.status_code == 200: print(f"Model {MODEL_ID} deleted successfully") else: print(f"Error: {response.json()}")
Node.js
const MODEL_ID = "00000000-1111-0000-1111-000000000000"; const response = await fetch( `https://api.erebine.ai/proj_ABC123/v1/models/${MODEL_ID}`, { method: "DELETE", headers: { "Authorization": "Bearer ere_myproject_your_api_key" } } ); if (response.ok) { console.log(`Model ${MODEL_ID} deleted successfully`); } else { const error = await response.json(); console.log(`Error: ${JSON.stringify(error)}`); }

Response

{ "id": "00000000-1111-0000-1111-000000000000", "object": "model", "deleted": true }

Revalidate Model

POST/proj_ABC123/v1/models/{modelId}/revalidate

Triggers re-validation of a model. The endpoint queues a revalidation job via the database service and returns status: "validating" immediately on success; the model row's status column is updated by the worker that picks up the job. Whether the request is accepted depends on the model's current status as enforced by the database service. Errors are surfaced as 404 Model not found.

Revalidation re-reads the model's config.json, README and LICENSE and refreshes the fields they supply, except a field you set through Update Model Metadata, which keeps your value until you clear it.

curl
curl -X POST https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000/revalidate \ -H "Authorization: Bearer ere_myproject_your_api_key"
Python
import requests MODEL_ID = "00000000-1111-0000-1111-000000000000" headers = {"Authorization": "Bearer ere_myproject_your_api_key"} response = requests.post( f"https://api.erebine.ai/proj_ABC123/v1/models/{MODEL_ID}/revalidate", headers=headers ) result = response.json() print(f"Status: {result['status']} - {result['message']}")
Node.js
const MODEL_ID = "00000000-1111-0000-1111-000000000000"; const response = await fetch( `https://api.erebine.ai/proj_ABC123/v1/models/${MODEL_ID}/revalidate`, { method: "POST", headers: { "Authorization": "Bearer ere_myproject_your_api_key" } } ); const result = await response.json(); console.log(`Status: ${result.status} - ${result.message}`);

Response

{ "id": "00000000-1111-0000-1111-000000000000", "status": "validating", "message": "Model validation queued" }

Share Model

POST/proj_ABC123/v1/models/{modelId}/share

Shares a model to the public catalog. The catalog role controls how other projects see the model.

Request Body

Parameter Type Description
catalog_roleoptional string One of deployable or shared. deployable (default) lists the model in the catalog so other projects can deploy their own copy. shared additionally lets shared-mode EIM endpoints run inference against the source project's deployment. Defaults to deployable when omitted.
curl
curl -X POST https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000/share \ -H "Authorization: Bearer ere_myproject_your_api_key" \ -H "Content-Type: application/json" \ -d '{"catalog_role": "deployable"}'
Python
import requests MODEL_ID = "00000000-1111-0000-1111-000000000000" headers = { "Authorization": "Bearer ere_myproject_your_api_key", "Content-Type": "application/json" } response = requests.post( f"https://api.erebine.ai/proj_ABC123/v1/models/{MODEL_ID}/share", headers=headers, json={"catalog_role": "deployable"} ) print(response.json())
Node.js
const MODEL_ID = "00000000-1111-0000-1111-000000000000"; const response = await fetch( `https://api.erebine.ai/proj_ABC123/v1/models/${MODEL_ID}/share`, { method: "POST", headers: { "Authorization": "Bearer ere_myproject_your_api_key", "Content-Type": "application/json" }, body: JSON.stringify({ catalog_role: "deployable" }) } ); console.log(await response.json());

Response

{ "id": "00000000-1111-0000-1111-000000000000", "catalog_role": "deployable", "is_shared": true, "shared_at": "2026-01-15T10:30:00Z" }

Unshare Model

POST/proj_ABC123/v1/models/{modelId}/unshare

Removes a model from the public catalog. After unsharing, the model is no longer discoverable or deployable by other projects. Catalog deployments that other projects already provisioned are not affected. A model that a model in another project uses as its speculative-decoding draft returns 409 counting those models; see Speculative Decoding.

curl
curl -X POST https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000/unshare \ -H "Authorization: Bearer ere_myproject_your_api_key"
Python
import requests MODEL_ID = "00000000-1111-0000-1111-000000000000" headers = {"Authorization": "Bearer ere_myproject_your_api_key"} response = requests.post( f"https://api.erebine.ai/proj_ABC123/v1/models/{MODEL_ID}/unshare", headers=headers ) print(response.json())
Node.js
const MODEL_ID = "00000000-1111-0000-1111-000000000000"; const response = await fetch( `https://api.erebine.ai/proj_ABC123/v1/models/${MODEL_ID}/unshare`, { method: "POST", headers: { "Authorization": "Bearer ere_myproject_your_api_key" } } ); console.log(await response.json());

Response

{ "id": "00000000-1111-0000-1111-000000000000", "is_shared": false }

Get Catalog Info

GET/proj_ABC123/v1/models/{modelId}/catalog-info

Returns catalog information for a model that has been shared to the public catalog. Returns 404 if the model is not in the catalog.

Response Fields

Field Type Description
id string Model id.
catalog_role string One of deployable or shared. See the Share Model section for the meaning of each.
deployment_count integer Number of projects that have deployed this model from the catalog.
is_on_shared_agents boolean Derived. true when catalog_role == "shared"; false otherwise.
is_public boolean Whether the catalog entry is discoverable by other projects.
featured boolean Editorial flag set by platform operators; not user-controlled.
shared_at string ISO-8601 timestamp of the first share.
curl
curl https://api.erebine.ai/proj_ABC123/v1/models/00000000-1111-0000-1111-000000000000/catalog-info \ -H "Authorization: Bearer ere_myproject_your_api_key"
Python
import requests MODEL_ID = "00000000-1111-0000-1111-000000000000" headers = {"Authorization": "Bearer ere_myproject_your_api_key"} response = requests.get( f"https://api.erebine.ai/proj_ABC123/v1/models/{MODEL_ID}/catalog-info", headers=headers ) if response.status_code == 200: info = response.json() print(f"Role: {info['catalog_role']}, Deployments: {info['deployment_count']}") else: print("Model is not in the catalog")
Node.js
const MODEL_ID = "00000000-1111-0000-1111-000000000000"; const response = await fetch( `https://api.erebine.ai/proj_ABC123/v1/models/${MODEL_ID}/catalog-info`, { headers: { "Authorization": "Bearer ere_myproject_your_api_key" } } ); if (response.ok) { const info = await response.json(); console.log(`Role: ${info.catalog_role}, Deployments: ${info.deployment_count}`); } else { console.log("Model is not in the catalog"); }

Response

{ "id": "00000000-1111-0000-1111-000000000000", "catalog_role": "deployable", "deployment_count": 5, "is_on_shared_agents": false, "is_public": true, "featured": false, "shared_at": "2026-01-15T10:30:00Z" }

Model Versions

Four endpoints manage model versions. Detailed semantics and request bodies are covered in the Model Versioning guide; the endpoint surface is summarized here.

Method Path Purpose
GET /proj_ABC123/v1/models/{modelId}/versions List all versions of a model.
POST /proj_ABC123/v1/models/{modelId}/versions Create a new version row for a model.
POST /proj_ABC123/v1/models/{modelId}/versions/{version}/promote Promote the named version to be the latest.
POST /proj_ABC123/v1/models/{modelId}/versions/{version}/rollback Roll back to the named version.

Preseeded Models

A model badged Preseeded in the dashboard was imported from a private EIM's local disk instead of uploaded. Its weights live only on the EIMs that imported them; the platform holds no copy and moves no bytes. Extended responses report this on the origin field, which is "preseeded" for these models and "uploaded" for everything else. The Private EIMs guide covers the import itself.

Metadata works the same as it does for an uploaded model. Update Model Metadata accepts the same fields, every section of the model's dashboard page is open, Hugging Face overrides included, and the values you save drive what the EIM launches the engine with: architecture, context length, quantization, and parsers all apply. The platform never harvests metadata out of the files, so what you enter is what the model reports.

The verbs that need platform-side bytes return 409 with the reason in the message: revalidate, share, unshare, and creating, promoting, or rolling back a version. A preseeded model has no catalog entry, no versions, and counts nothing against your project's storage quota. Delete returns 409 while an endpoint or an anchor endpoint binds the model; once unbound, the delete removes the row and every holding EIM purges its local copy. Importing the same directory again afterwards creates a new model with a new id.

Error Responses

Every endpoint in this surface returns OpenAI-shaped error envelopes. The error classes below cover the failure modes you are likely to hit when integrating.

401 Unauthorized

Returned when no Authorization header is present or the bearer token is malformed.

{ "error": { "message": "Missing or invalid Authorization header", "type": "invalid_request_error", "code": "missing_authorization" } }

403 Forbidden

Returned when the API key is valid but belongs to a different project than the one in the URL path, or when the key has been revoked.

{ "error": { "message": "API key does not authorize access to this project", "type": "invalid_request_error", "code": "project_mismatch" } }

404 Model not found

GET /models/{modelId} answers an id it cannot find -- not a model id, no such model, or a model the calling project cannot see -- with the body the OpenAI API sends for a model the caller cannot use, which names the id you sent. DELETE /models/{modelId}, the route an OpenAI client's models.delete() calls, answers a model the calling project does not own with the same body.

{ "error": { "message": "The model `gpt-4o-mini` does not exist or you do not have access to it.", "type": "invalid_request_error", "param": null, "code": "model_not_found" } }

The other /models/{modelId} endpoints are model management the OpenAI API does not have. PATCH /models/{modelId}, revalidate, GET versions and catalog-info answer a model id that does not exist in the calling project with the body below; catalog-info answers the same way, with the message Model is not in the catalog, when the model exists but is not in the public catalog, and unshare with the message Model not found or not shared.

{ "error": { "message": "Model not found", "type": "not_found_error", "code": "not_found" } }

400 Invalid field value

Returned by PATCH /models/{modelId} when a key carries a value the field does not accept: an unknown quantization method, dtype, pre-quantization method or capability tag, a context length below 128, an empty or unknown workload tag, a null for a field that cannot be cleared, or a negative dimension. param names the key, the message names the value, and nothing of the body is stored.

{ "error": { "message": "Invalid quantization method: fp4", "type": "invalid_request_error", "code": "invalid_request", "param": "quantization" } }

403 Engine setting without the management scope

Returned by PATCH /models/{modelId} when the body changes an engine setting and the API key does not carry the management scope. A change to trust_remote_code by a key whose user is not the project owner returns 403 with param trust_remote_code.

{ "error": { "message": "API key requires 'management' scope to change a model's engine settings (context_length).", "type": "permission_error", "code": "scope_insufficient", "param": "context_length" } }

422 Invalid HF config overrides

Returned by PATCH /models/{modelId} and by the override preview when the rows do not assemble: an unknown value type, a bad literal, a duplicate path, a path that is a prefix of another, an array gap, or a set too large. Also returned when overrides are set on a model whose speculative decoding is enabled. param is hf_overrides.

400 Invalid speculative decoding setting

Returned by PATCH /models/{modelId} when the merged speculative_decoding setting breaks a rule: enabled without a method, a method that runs a draft with no draft_model_id, a draft on mtp or ngram, ngram_fallback on a method other than mtp, a value out of range, a draft that is not usable (see The draft model), or speculation on a model that keeps HF config overrides after the update. The message names the rule and nothing is stored.

409 Model is a speculative decoding draft

Returned by DELETE /models/{modelId} when another model uses this one as its speculative decoding draft. The message names those models.

400 Invalid pagination bound

Returned by the management listing GET /proj_ABC123/v1/models when limit or offset carries a negative value; the bound is refused rather than clamped. The param field names the offending parameter; limit is checked first, so a request that makes both negative is told about limit. The inference-facing listings do not share this refusal -- a negative bound there is ignored and the full visible inventory is returned.

{ "error": { "message": "Invalid value for 'limit': must not be negative", "type": "invalid_request_error", "code": "invalid_request", "param": "limit" } }

Keyboard Shortcuts

This page registers a small set of chord shortcuts for jumping between endpoints. Shortcuts are suppressed while typing in inputs, textareas, or contenteditable surfaces. Press ? to toggle the help dialog.

Chord Action
g oJump to Overview
g bJump to Base URL
g lJump to List Models
g dJump to Get Model Details
g uJump to Update Model
g xJump to Delete Model
g rJump to Revalidate Model
g sJump to Share Model
g nJump to Unshare Model
g cJump to Catalog Info
g vJump to Model Versions
g pJump to Preseeded Models
g eJump to Error Responses
g tJump to Scope Table
?Toggle this shortcuts dialog
EscClose the shortcuts dialog

Endpoint Scope Table

All Model Management endpoints below are mounted on the project inference route group and gated by the same project API-key middleware as /v1/chat/completions. There is no separate models or Model Management scope; any valid project API key is accepted, except that a PATCH changing an engine setting needs the management scope.

Method Path Route group Auth
GET/v1/modelsinferenceproject API key
GET/v1/models/{modelId}inferenceproject API key
PATCH/v1/models/{modelId}inferenceproject API key; management scope for engine settings
GET/v1/models/{modelId}/hf-overridesinferenceproject API key
POST/v1/models/{modelId}/hf-overrides/previewinferenceproject API key
DELETE/v1/models/{modelId}inferenceproject API key
POST/v1/models/{modelId}/revalidateinferenceproject API key
POST/v1/models/{modelId}/shareinferenceproject API key
POST/v1/models/{modelId}/unshareinferenceproject API key
GET/v1/models/{modelId}/versionsinferenceproject API key
POST/v1/models/{modelId}/versionsinferenceproject API key
POST/v1/models/{modelId}/versions/{version}/promoteinferenceproject API key
POST/v1/models/{modelId}/versions/{version}/rollbackinferenceproject API key
GET/v1/models/{modelId}/catalog-infoinferenceproject API key