erectl endpoints
Manage inference endpoints from a shell: list and inspect them, create and patch them, provision and unprovision, configure CORS and health probes, and attach several Erebine Inference Microservices (EIMs) to one endpoint with per-EIM traffic weights. Project-scoped and context-aware.
Overview
An endpoint is a routable model deployment in your project. The
endpoints command group covers its full lifecycle. Every
subcommand resolves the active erectl context and authenticates
with the project's API key; there are no project UUIDs to pass. Use
--context <name> to target a specific deployment.
An endpoint can be served by one or more EIMs. When it has several, requests without a warm prompt are apportioned across them by weight, and the endpoint keeps serving as long as at least one attached EIM is live. The eims subcommand manages that membership.
erectl endpoints # list (default action)
erectl endpoints show my-endpoint
erectl endpoints eims list my-endpoint
endpoints list
List the project's endpoints. The default action when no subcommand is given.
erectl endpoints
erectl endpoints -o wide
erectl endpoints -o json --columns id,status,model --filter status=ready
Honors the shared output flags (-o, --columns, --filter, --sort-by, --watch). Output columns: ID, NAME, SLUG, MODEL, STATUS, TIER, TASK.
STATUS says whether the endpoint is serving: active when an EIM serves it, provisioning while its EIM is coming up, disconnected when nothing is placed, failed or unrecoverable when the placement failed, draining while it is taken off its EIM, or the status a disabled endpoint was set to (suspended, draining, error). -o json carries it as effective_status beside status, which is only whether the endpoint is enabled, and the reason as effective_status_reason when there is one.
endpoints show
Show the full detail of one endpoint, including its provisioning state and primary EIM. Status: is whether it is serving, as in the list; when nothing can serve it because every EIM of its shared model has refused it or is parked, a Status Reason: line says why; when the status differs from the one endpoints update --status sets, a Lifecycle: line shows the latter.
erectl endpoints show <endpoint-id>
endpoints create
Create an endpoint.
erectl endpoints create \
--name "My Endpoint" \
--slug my-endpoint \
--model-id <model-uuid> \
--tier-id self_hosted
| Flag | Description |
|---|---|
--name | Display name. |
--slug | Lowercase kebab-case slug (unique per project). |
--model-id | Model UUID. |
--tier-id | Service tier identifier. |
--task-mode | generate (the default), embed, score, transcribe, speech, realtime, image_generation or systemone. An image generation model is served by image_generation only; see Image Generation. |
--required-gpu-model, --required-gpu-count | GPU requirements. |
--quantization-override | Quantization override. |
A new endpoint is provisioning, and nothing serves it, until one of the project's EIMs is attached with endpoints eims add, which places the model on that EIM, or until it is set active with endpoints update <endpoint-id> --status active, which places it automatically on its tier's EIMs. The create prints both commands with the new endpoint's id. Attach one EIM or many; each takes a share of traffic.
endpoints update
Patch an endpoint. Only the flags you pass are changed.
erectl endpoints update <endpoint-id> --name "Renamed"
erectl endpoints update <endpoint-id> --status suspended
Accepts --name, --tier-id, --status (active|suspended|draining), --rpm, --tpm, GPU flags, --quantization-override, --task-mode.
endpoints delete
Soft-delete an endpoint. Prompts for confirmation unless --force is given.
erectl endpoints delete <endpoint-id>
erectl endpoints delete <endpoint-id> --force
endpoints provision
Send the provision command to every EIM attached to the endpoint. endpoints eims add already places the model on the EIM it attaches, so use this to retry a placement that failed. An EIM already serving the endpoint is left alone; any other attached EIM is sent the command again. An endpoint that has exhausted its startup attempts is refused with endpoint_unrecoverable (409): fix the cause, then ask an administrator to reschedule it.
erectl endpoints provision <endpoint-id>
The command prints the provisioning request id the router dispatched; -o json prints {"provisioning_id": ...}.
Fails with endpoint_has_no_agents (409) when no EIM is attached, with endpoint_not_active (409) when the endpoint is suspended, draining or in error (set it active first with endpoints update --status active), and with conflicting_endpoint (409) when an attached EIM serves another endpoint.
endpoints unprovision
Unprovision an endpoint from its attached EIMs. Each EIM unloads the model before it is detached.
erectl endpoints unprovision <endpoint-id>
The command prints the unprovisioning request id; -o json prints {"unprovisioning_id": ...}.
The endpoint stays active. With no EIM attached it falls back to automatic placement on its tier, so the model may be placed again on any eligible EIM, including one just released. To stop serving the endpoint, suspend it: erectl endpoints update <endpoint-id> --status suspended.
endpoints regenerate-capabilities
Re-run capability synthesis for an endpoint. Replies once the new capabilities have been written; a synthesis already in flight, or a deployment with no synthesizer configured, is refused with a conflict.
erectl endpoints regenerate-capabilities <endpoint-id>
endpoints cors
Read or set the endpoint's CORS configuration.
erectl endpoints cors get <endpoint-id>
erectl endpoints cors set <endpoint-id> --allow-origins "https://app.example.com"
endpoints health
Read or set the health-probe configuration, or trigger a synthetic probe.
erectl endpoints health config get <endpoint-id>
erectl endpoints health config set <endpoint-id> ...
erectl endpoints health check <endpoint-id>
Multi-EIM endpoints
A self-hosted endpoint can be served by several EIMs at once. Traffic is split across them by a per-EIM weight, and the endpoint stays up while any one attached EIM is live. Setting an EIM's weight to 0 drains it: it takes no new requests, in-flight requests finish, and its model stays loaded so the EIM can be un-drained instantly.
The warm cache comes before the weights. A request whose prompt prefix an
attached EIM already holds, and a conversation sent with
X-Conversation-Id, go to that EIM ahead of the split; a
drained EIM takes no new requests either way. See
Router: Prefix-Cache Affinity.
The endpoints eims subcommand manages that membership. Adding an
EIM places the model on it; removing an EIM tells it to unload the model
first, then detaches it. An EIM serves exactly one endpoint at a time, so
attaching an EIM already serving another endpoint is rejected.
erectl endpoints eims list <endpoint-id>
erectl endpoints eims add <endpoint-id> --eim <eim-uuid> --weight 300
erectl endpoints eims weight <endpoint-id> <worker-id> --weight 0
erectl endpoints eims remove <endpoint-id> <worker-id>
EIM membership on anchor (shared, admin-owned) endpoints is managed from the admin dashboard, not this command group.
endpoints eims list
List the EIMs attached to an endpoint, with each EIM's weight and live state.
erectl endpoints eims list <endpoint-id>
Output columns: WORKER_ID, WEIGHT, STATE. A weight of 0 reads as draining.
endpoints eims add
Attach an EIM to the endpoint. The model is placed on the EIM automatically; the EIM then joins the weighted rotation.
erectl endpoints eims add <endpoint-id> --eim <eim-uuid>
erectl endpoints eims add <endpoint-id> --eim <eim-uuid> --weight 300
| Flag | Description |
|---|---|
--eim | EIM UUID to attach. Required. |
--weight | Relative traffic weight (default 100). 0 drains the EIM. |
Fails with agent_busy (400) if the EIM already serves another endpoint. The attach also refuses an EIM whose engine cannot serve the endpoint (engine_incompatible, 422) and one the model does not fit (model_incompatible, 400).
The command prints the endpoint's primary EIM after the attach. -o json prints {"id": <eim-uuid>, "status": "attach_requested", "endpoint_id", "weight", "primary_worker_id"}; weight appears when --weight was passed.
endpoints eims weight
Change an attached EIM's traffic weight. Takes effect immediately: a weight of 0 drains the EIM (no new requests; in-flight requests finish; model stays loaded).
erectl endpoints eims weight <endpoint-id> <worker-id> --weight 300
erectl endpoints eims weight <endpoint-id> <worker-id> --weight 0 # drain
| Argument / Flag | Description |
|---|---|
<worker-id> | The attached EIM's worker id (from eims list). |
--weight | New relative weight (>= 0). Required. |
-o json prints {"id": <worker-id>, "status": "weight_set", "endpoint_id", "weight", "primary_worker_id"}.
endpoints eims remove
Detach an EIM from the endpoint. The EIM is told to unload the model first, then the binding is removed; if the EIM cannot be reached the binding is kept and the detach retried.
erectl endpoints eims remove <endpoint-id> <worker-id>
The command prints the endpoint's primary EIM after the detach, none when no EIM is left. -o json prints {"id": <worker-id>, "status": "detach_requested", "endpoint_id", "primary_worker_id"}, without primary_worker_id when none is left.
Removing the last remaining EIM leaves the endpoint active: it falls back to automatic placement on its tier and may be placed again on any eligible EIM. To stop serving, suspend the endpoint with endpoints update --status suspended. To drain an EIM without detaching it, set its weight to 0 instead.
Examples
Serve one endpoint from two EIMs, weighting a faster GPU higher, then drain one for maintenance:
# attach two EIMs, 3:1 split toward the faster box
erectl endpoints eims add my-endpoint --eim 1f0e-...-a1 --weight 300
erectl endpoints eims add my-endpoint --eim 2a7b-...-c9 --weight 100
# confirm the rotation
erectl endpoints eims list my-endpoint
# drain the slower EIM for maintenance (traffic shifts to the other)
erectl endpoints eims weight my-endpoint DFW3:gpu-2a7b --weight 0
# maintenance done -- un-drain instantly
erectl endpoints eims weight my-endpoint DFW3:gpu-2a7b --weight 100