erectl models
Full model lifecycle from the terminal. List, inspect, update every metadata field, version, delete, share to the catalog, revalidate, probe capabilities, and ask the router for workspace-fitted recommendations. One flag per action; bulk delete the only multi-id verb.
// requires an active erectl deployment and an API key with inference scope.
Overview#
The erectl models command group provides full lifecycle management for
models stored in your project. It follows the unified command pattern used across
all erectl resource commands:
erectl models # List all models
erectl models <MODEL_ID> # Show model details
erectl models <MODEL_ID> --<action> # Perform an action on a model
erectl models update <MODEL_ID> [options] # Update metadata
erectl models hf-overrides <MODEL_ID> # Show HF config overrides
erectl models versions <MODEL_ID> # Manage model versions
Only one action flag per invocation, and multiple IDs only with
--delete (bulk). Metadata changes go through
erectl models update.
Back to erectl CLI overview.
List Models#
// try this first
Run erectl models with no arguments to print the table of IDs you can substitute into every other example on this page. Append --help to any subcommand for the in-CLI reference.
Omit all IDs to list models in your project. The default table output includes ID, Name, Format, Size, Status, Version, and Shared columns.
# Basic listing
erectl models
# Wide table output (adds Architecture, Quantization, Context, Parameters, Created)
erectl models -o wide
# Request extended fields on the JSON response (no effect on text tables)
erectl models --extended -o json
# Paginate results
erectl models --limit 10 --offset 20
# JSON output
erectl models -o json
Listing Options
| Option | Description |
|---|---|
--limit <n> |
Maximum number of results. The router applies a server-side ceiling of 100 (and uses 100 as the fallback when no value is supplied); larger values are accepted by the CLI but silently truncated to 100 by the API. |
--offset <n> |
Pagination offset. When more results exist, a hint is printed at the end of the table. |
--extended |
Request extended fields (architecture, quantization, context length, parameter count, creation time) from the API. Only changes the response payload when combined with -o json; the default text table is unaffected. To see the extra columns in a table, use -o wide. |
Get Model#
Provide a single model ID to display its full details.
# Human-readable key-value output
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01
# Show full description without truncation
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --full
# JSON output
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 -o json
The detail view includes ID, Name, Created, Owner, Format, Size, Status, Version,
Is Latest, Description, Architecture, Quantization, Context Length, Parameters,
Workload Type, Shared, Catalog Role, Origin, Trust Remote Code, License, Hugging
Face ID, Capabilities, the two parsers, Layers, Hidden Size, Vocab Size, Torch
Dtype, the expert counts, Pre-Quantized, Reasoning Efforts, Engine Tuning (the
settings that are not auto, or auto), and, for a model you own,
Speculative Decoding and the number of HF config override rows.
The Capabilities line states whether structured output is
proven by probe or not proven: only the
capability probe sets that tag, and a model the probe found without it
refuses a json_schema request. json_object
carries no schema and is accepted on every chat model.
The description is truncated to 120 characters by default. Use --full
to display it in its entirety and to print the assembled HF config overrides.
Update Metadata#
erectl models update <MODEL_ID> changes only the fields you
pass; every other field keeps its value. none clears a field, and
the parsers reset with auto. A value the server refuses is named in
the error, and nothing of the call is stored.
# Update description and context length
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 \
--description "Fine-tuned for code generation" \
--context-length 8192
# Set the capability tags and the parsers
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 \
--capabilities chat,vision \
--tool-call-parser hermes \
--reasoning-parser auto
# Update the dimensions, then clear one that was wrong
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 \
--parameter-count 7000000000 --hidden-size 4096 --num-layers 32 --vocab-size 32000
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --vocab-size none
# Mark as pre-quantized
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 \
--pre-quantized \
--pre-quantization-method awq \
--pre-quantization-bits 4
# Tune the vLLM launch, then return one setting to auto
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 \
--max-num-seqs 128 --gpu-memory-utilization 0.9 --prefix-caching off
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --prefix-caching auto
# Print the request body without sending it
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --license MIT --dry-run
Engine settings are the fields an engine launches with: quantization, context
length, the parsers, trust remote code, the
engine tuning, capabilities, the dimensions
except the vocabulary size, the torch dtype, the expert counts, the
pre-quantization fields, speculative decoding and the HF config overrides.
Changing one needs an
API key with the management scope, and every endpoint serving the
model is relaunched with the new value. Passing a setting's current value
changes nothing and needs no extra scope. Changing trust remote code also needs
the project owner.
A field you set is kept across revalidation: Revalidate
re-reads the model's config.json, README and LICENSE only for the
fields you have not edited. Clearing a field with none hands it
back: the next revalidation fills it from those files again.
Listing Options
| Option | Type | Description |
|---|---|---|
--description <text> |
string | Markdown description of the model. "" clears it. |
--description-file <path|-> |
path | Read the description from a file, or from standard input with -. |
--license <text|none> |
string | License identifier (e.g., "MIT", "apache-2.0"). |
--hugging-face-id <owner/name|none> |
string | Upstream Hugging Face repository the weights come from. |
--workload-type <tag,...> |
string | Workload tags: chat, code, reasoning, embedding, multilingual. At least one. |
--reasoning-efforts <detected|unsupported|list> |
string | reasoning_effort values the model accepts: detected (reset to the values read from the model's reasoning parser, chat template and family), unsupported (the model does not reason), or a comma-separated list. default is accepted as another name for detected. |
--architecture <text|none> |
string | Model architecture (e.g., "qwen3"). |
Engine Options
| Option | Type | Description |
|---|---|---|
--capabilities <tag,...|none> |
string | Capability tags, replacing the whole list: chat, vision, video_input, audio_input, transcription, speech, realtime, forced_tool_choice. Requests are admitted per tag. structured_output is set only by the capability probe and is kept whatever you send. |
--quantization <method|native> |
string | Runtime quantization method applied by the serving backend. native serves the weights as stored. |
--context-length <n> |
integer | Maximum context length in tokens (vLLM --max-model-len), at least 128. It cannot be cleared. |
--tool-call-parser <name|auto> |
string | vLLM --tool-call-parser. auto detects it from the architecture. |
--reasoning-parser <name|auto> |
string | vLLM --reasoning-parser. auto detects it from the architecture. |
--[no-]trust-remote-code |
flag | Whether the engine may execute code shipped in the model bundle. Project owners only. |
--scheduling-policy <priority|fcfs> |
string | vLLM --scheduling-policy. priority (the default) serves realtime and interactive requests first; fcfs serves requests in arrival order. |
--kv-cache-dtype <dtype> |
string | vLLM --kv-cache-dtype. auto (the default) passes no flag; any other value is passed as given. An endpoint can override it. |
--tool-narrowing <default|on|off> |
string | Chat tool narrowing for the model. on starts its chat turns with a core of tools and a catalog the model loads the rest from; off never narrows it; default follows the platform setting. See Chat Options. |
--steering-role <default|user_note> |
string | How the router's mid-turn corrections reach the model. user_note sends them as a platform note after the tool results, which keeps the prompt cache; default sends system instructions. Refused, with the reason, on a model whose chat template does not take a note. erectl models show prints both settings. |
--torch-dtype <dtype|none> |
string | Native PyTorch dtype: float16, bfloat16, or float32. |
--[no-]pre-quantized |
flag | Whether the weights are stored in quantized form. |
--pre-quantization-method <m|none> |
string | Pre-quantization algorithm: gptq, awq, compressed-tensors, or fp8. |
--pre-quantization-bits <4|8|none> |
integer | Bit width of the pre-quantization. |
Dimension Options
| Option | Type | Description |
|---|---|---|
--parameter-count <n|none> |
integer | Total number of model parameters; sizes the VRAM estimate. |
--hidden-size <n|none> |
integer | Hidden layer dimensionality. |
--num-layers <n|none> |
integer | Number of transformer layers. |
--vocab-size <n|none> |
integer | Vocabulary size. |
--num-experts <n|none> |
integer | Total number of MoE experts. |
--num-experts-per-tok <n|none> |
integer | Number of experts activated per token (MoE models). |
Engine Tuning Options
vLLM launch settings for every engine serving the model. Each is
auto unless set: the Erebine Inference Microservice (EIM) chooses the value as it
does for any model. auto and none both return a setting to auto.
A value the operator sets on an EIM, by option, environment variable or
vLLM argument, wins over the model's on that EIM.
| Option | Type | Description |
|---|---|---|
--expert-parallel <on|off|auto> |
string | vLLM --enable-expert-parallel for a mixture-of-experts model on more than one GPU. Auto turns it on there. |
--enable-eplb <on|auto> |
string | vLLM --enable-eplb, expert parallel load balancing. Applies with expert parallel across more than one rank on NVIDIA CUDA. Refused with --expert-parallel off. |
--all2all-backend <backend|auto> |
string | vLLM --all2all-backend under expert parallel: allgather_reducescatter, flashinfer_nvlink_two_sided or flashinfer_nvlink_one_sided. |
--gpu-memory-utilization <fraction|auto> |
number | vLLM --gpu-memory-utilization, above 0 and at most 1. |
--max-num-seqs <n|auto> |
integer | vLLM --max-num-seqs, 1 or more. |
--max-num-batched-tokens <n|auto> |
integer | vLLM --max-num-batched-tokens, 1 or more and at least --max-num-seqs when both are set. |
--long-prefill-token-threshold <n|auto> |
integer | vLLM --long-prefill-token-threshold, 0 or more; 0 is no cap. A value above the context length is not sent. |
--enforce-eager <on|auto> |
string | vLLM --enforce-eager: run without CUDA graphs. |
--prefix-caching <on|off|auto> |
string | vLLM automatic prefix caching. Auto keeps it on; audio models always run without it. |
--async-scheduling <off|auto> |
string | vLLM async scheduling. off passes --no-async-scheduling. A model cannot force it on. |
--disable-custom-all-reduce <on|auto> |
string | vLLM --disable-custom-all-reduce: every all-reduce across GPUs uses NCCL. |
--engine-config <flag=json|flag=none> |
string | One vLLM JSON config: attention_config, compilation_config, speculative_config or reasoning_config, set to one JSON object, which is merged over the object the EIM derives for that flag (a key set to null removes the EIM's value). flag=none clears that flag. Repeat it; flags left out keep their objects. A key that no vLLM release the EIM images ship accepts is refused. |
--clear-engine-configs |
flag | Clear every vLLM JSON config. |
# Set one vLLM JSON config, then clear it
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --engine-config attention_config='{"indexer_kv_dtype":"mxfp4"}'
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --engine-config attention_config=none
HF Config Override Options
| Option | Type | Description |
|---|---|---|
--hf-override <key.path=type:value> |
string | One override row; type is string, number or boolean. Repeat it. The rows replace the whole set. |
--hf-overrides-file <rows.json> |
path | Replace the set with the rows in a file, in the shape erectl models hf-overrides -o json prints. |
--clear-hf-overrides |
flag | Remove every override row. |
Overrides and an enabled speculative decoding setting refuse each other. One call may clear the overrides and enable speculation; it may not set both.
File Options
| Option | Type | Description |
|---|---|---|
--metadata <file.json|-> |
path | Apply a whole update body from a file (see Metadata File). A field set both in the file and by an option is refused. |
Speculative Decoding Options
These change the model's speculative decoding setting. Only the options you pass change; the rest of the setting is kept. The server checks the result as a whole and refuses a setting that breaks a rule, naming it. The setting, its methods and its rules are described in Model Management.
| Option | Type | Description |
|---|---|---|
--speculative on|off |
string | Speculate by default on every endpoint of the model. on needs a method. |
--speculative-method <method>|none |
string | mtp, ngram, eagle, eagle3, dflash, draft_model, medusa, or mlp_speculator. none clears it. |
--speculative-tokens <n>|default |
integer | Tokens drafted per step, 1 to 32. default uses the method's default. |
--draft-model <id>|none |
string | Id of the catalog model that drafts. Required by eagle, eagle3, dflash, draft_model, medusa and mlp_speculator; refused by mtp and ngram. none clears it. |
--ngram-fallback on|off |
string | mtp only: speculate with ngram when the checkpoint ships no multi-token-prediction layers. |
--speculative-temperature <x>|none |
number | Default temperature (0 to 2) for requests that send none while the model speculates. |
--speculative-top-p <x>|none |
number | Default top_p (above 0, at most 1) for requests that send none while the model speculates. |
--speculative-top-k <n>|none |
integer | Default top_k (1 or more) for requests that send none while the model speculates. A top_k default lets a thinking budget run under speculative decoding; see Sampling defaults. |
--speculative-reset |
flag | Reset the whole setting: off, no method, no draft, no sampling defaults. Takes no other speculative option. |
The model details (erectl models <id>) show the setting on the
Speculative Decoding line.
Metadata File#
A metadata file is the body of
PATCH /v1/models/{id}:
a JSON object with snake_case keys, where an absent key leaves the field alone and
null clears it. A key the request body does not define is refused by
name, so a typo never goes unnoticed. The same file works with
erectl models update --metadata and with
erectl upload --metadata.
A speech model's voice settings ride under speech: an object with
disabled_voices, default_voice and aliases.
Absent leaves the settings alone, null clears them, and an object
replaces them whole. A voice the model does not offer is refused by name; see
Speech Voices.
{
"speech": {
"disabled_voices": ["uncle_fu"],
"default_voice": "serena",
"aliases": {"alloy": "serena", "echo": "ryan"}
}
}
{
"description": "Qwen3 8B instruct, fine-tuned for code.",
"license": "apache-2.0",
"workload_type": "chat,code",
"capabilities": ["chat", "vision"],
"context_length": 32768,
"tool_call_parser": "hermes",
"reasoning_efforts": ["low", "medium", "high"],
"torch_dtype": null,
"hf_overrides": [
{"key_path": "rope_scaling.factor", "value_type": "number", "value": "4.0"}
]
}
# Apply the file
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --metadata model.json
# Read it from standard input, and add one option the file does not set
cat model.json | erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --metadata - --no-trust-remote-code
HF Config Overrides#
erectl models hf-overrides <MODEL_ID> lists the Hugging Face
config override rows stored on a model you own. With --preview it
assembles the rows in a file into the document the engine receives as
--hf-overrides, and stores nothing.
# List the rows
erectl models hf-overrides 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01
# Save them, edit the file, preview it, then store it
erectl models hf-overrides 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 -o json > rows.json
erectl models hf-overrides 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --preview rows.json
erectl models update 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --hf-overrides-file rows.json
| Option | Description |
|---|---|
--preview <rows.json|-> |
Assemble the rows in a file ({"rows": [...]}) without storing them. A set the server refuses is named in the error. |
Delete a Model#
Delete a single model or multiple models in bulk. A confirmation prompt is
shown unless --force is supplied. Use --dry-run to
preview the operation without making changes.
# Delete with confirmation prompt
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --delete
# Skip confirmation
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --delete --force
# Bulk delete (multiple IDs, only supported with --delete)
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 2a3c5b6e-7d8f-4019-9c2a-1b4d3e5f6a7c 3f7e8d9a-1c2b-4d5e-8f60-7a8b9c0d1e2f --delete --force
# Preview without deleting
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --delete --dry-run
Bulk deletion prints a per-model result ([OK] / [FAIL])
and a summary line showing how many succeeded. With -o json, one id
or several print the same document: an array with
{"id", "status"} per id, in argument order, where
status is deleted or failed. The command
exits 1 if any model was not deleted, and the reason goes to standard error.
A model that another model uses as its speculative decoding draft is not
deleted: the server answers 409 naming those models. Change or
clear their --draft-model first.
// see also REST: DELETE /v1/models/{id}
Revalidate#
Trigger a re-check of the model's files and update its validation status. This is useful after a file has been corrected or a partial upload completed.
# Queue revalidation and return immediately
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --revalidate
# Queue revalidation and poll until a terminal status is reached
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --revalidate --wait
# Set a custom wait timeout (default: 600 seconds)
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --revalidate --wait --wait-timeout 300
Revalidation Options
| Option | Description |
|---|---|
--revalidate |
Trigger model revalidation. Requires a single model ID. |
--wait |
Poll until revalidation reaches a terminal state (active, ready, validated, error, failed, or invalid). Only valid with --revalidate. |
--wait-timeout <seconds> |
Maximum seconds to wait when --wait is used. Default: 600. |
Version Management#
The models versions subcommand manages semantic versions for a model.
Versions track deployment history and allow rollback to a previous state.
# List all versions for a model
erectl models versions 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01
# Create a new version (semver string required)
erectl models versions 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --create 2.0.0
erectl models versions 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --create 2.0.0 --notes "Improved accuracy on code tasks"
# Promote a version to latest
erectl models versions 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --promote 2.0.0
# Rollback to a previous version
erectl models versions 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --rollback 1.0.0
Version Options
| Option | Description |
|---|---|
--create <version> |
Create a new version using a semver string (e.g., 2.0.0). |
--promote <version> |
Mark the given version as the latest active version. |
--rollback <version> |
Roll back to a previously created version. |
--notes <text> |
Version release notes. Only valid with --create. |
Only one action (--create, --promote, or
--rollback) may be specified per invocation.
The version list table shows: Version, ID, Latest, Status, Notes, and Created.
// see also Model Versioning for promotion semantics and rollback safety.
Options Reference#
Full reference for all flags and options accepted by erectl models. The options of erectl models update are listed under Update Metadata, including the speculative decoding options and the engine tuning options.
Action Flags (mutually exclusive)
| Flag | State | Requires ID | Description |
|---|---|---|---|
--delete |
destructive | Yes (one or more) | Delete one or more models. Prompts for confirmation unless --force is set. |
--share |
reversible | Yes (one) | Share the model to the catalog. |
--unshare |
reversible | Yes (one) | Remove the model from the catalog. |
--revalidate |
safe | Yes (one) | Trigger model file revalidation. |
--catalog-info |
read-only | Yes (one) | Display catalog information for the model. |
Modifier Flags
| Flag | Description |
|---|---|
--force |
Skip the confirmation prompt on destructive operations. |
--full |
Show the full description instead of truncating to 120 characters, and the assembled HF config overrides. |
--extended |
Request extended fields on the API response. Affects JSON output only; use -o wide to render extra columns in the text table. |
--wait |
Poll until revalidation completes. Only valid with --revalidate. |
--wait-timeout <n> |
Timeout in seconds for --wait. Default: 600. |
--limit <n> |
Maximum number of results when listing. The router enforces a server-side ceiling of 100 and uses 100 as the fallback when omitted. |
--offset <n> |
Pagination offset when listing. |
Examples#
Update and Revalidate a Model
MODEL_ID="8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01"
# Set metadata
erectl models update "$MODEL_ID" \
--description "Qwen3 7B instruct, fine-tuned for code" \
--architecture "qwen3" \
--context-length 32768 \
--parameter-count 7000000000 \
--workload-type "chat,code"
# Revalidate and wait for result
erectl models "$MODEL_ID" --revalidate --wait
Speculative Decoding with a Draft Model
MODEL_ID="8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01"
DRAFT_ID="2a3c5b6e-7d8f-4019-9c2a-1b4d3e5f6a7c"
# DFlash with 7 speculative tokens; the EIM syncs the draft itself
erectl models update "$MODEL_ID" \
--speculative on \
--speculative-method dflash \
--speculative-tokens 7 \
--draft-model "$DRAFT_ID"
# Let thinking budgets run while speculating
erectl models update "$MODEL_ID" --speculative-top-k 20
# Native MTP, falling back to n-gram when the checkpoint has no MTP layers
erectl models update "$MODEL_ID" --speculative-method mtp --draft-model none --ngram-fallback on
# Turn it off, keeping the rest of the setting; or reset it entirely
erectl models update "$MODEL_ID" --speculative off
erectl models update "$MODEL_ID" --speculative-reset
Version Promotion Workflow
MODEL_ID="8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01"
# Create new version after fine-tuning
erectl models versions "$MODEL_ID" \
--create 2.1.0 \
--notes "Reduced hallucination rate on factual queries"
# Verify the version list
erectl models versions "$MODEL_ID"
# Promote once tested
erectl models versions "$MODEL_ID" --promote 2.1.0
Bulk Delete Old Models
# Preview first
erectl models 4a5b6c7d-8e9f-4012-9a3b-5c6d7e8f9012 5b6c7d8e-9f01-4234-8a4b-6c7d8e9f0123 6c7d8e9f-0123-4456-8b5c-7d8e9f012345 --delete --dry-run
# Execute
erectl models 4a5b6c7d-8e9f-4012-9a3b-5c6d7e8f9012 5b6c7d8e-9f01-4234-8a4b-6c7d8e9f0123 6c7d8e9f-0123-4456-8b5c-7d8e9f012345 --delete --force
Share to Catalog and Check Info
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --share
erectl models 8b1f4c2d-9e7a-4f3b-bc11-2d8e6a4c5f01 --catalog-info