Private Agents

Overview

For a side-by-side of private agents vs. shared agents, see the Agent Types overview.

This page documents the project-scoped Frontend API (authenticated with a project API key or dashboard session). The router also exposes a parallel API-key-authed management surface at /:project_id/v1/management/join-keys and /:project_id/v1/management/agents; see the API reference for details. Defaults may differ between the two surfaces.

Agent Fields

The following fields describe an enrolled agent. They are returned by the agent list and detail API endpoints.

Join Key Management

Join keys are secure tokens that authorize a backend agent to enroll in your project. Each key is tied to a specific project, carries a region assignment, and expires after a configurable TTL (maximum 1 hour). Keys can be single-use or multi-use.

API Endpoints

Join Key Fields

The following fields are accepted when creating a join key (POST).

The response for a create request includes the full key value exactly once. It cannot be retrieved later because only the SHA-256 hash is stored.

The list and detail responses include the following join key fields:

Join Key API Examples

In the examples below, replace proj_ABC123 with your project external id (e.g. proj_a1b2c3d4). The project external id is the leading path segment, and the API key must carry the management scope.

Create a Join Key

List Join Keys

Revoke a Join Key

Agent API

Agents are enrolled automatically using a join key (see Agent Enrollment). Once enrolled, you can list, view, update, and remove agents via the following endpoints.

PATCH Request Body

All fields are optional. Only provided fields are updated.

Agent API Examples

List Agents

Get Agent Details

Suspend an Agent

Resume an Agent

Get Agent Events

Remove an Agent

Agent Enrollment

To enroll a private agent:

1. Create a join key in the dashboard or via the API (see Join Key API Examples).
2. Save the full key value immediately, it is shown only once.
3. Run the EIM agent with the enroll subcommand, passing the join key.
4. The agent authenticates with the router and establishes a secure encrypted connection.
5. The agent transitions to active status and begins sending heartbeats. It is then ready to receive inference requests.

After enrollment, start the agent in serve mode to begin handling inference requests. See the EIM Deployment guide for detailed setup instructions.

Preseeding a Local Model

A preseeded model is one that is already on the agent's disk. Point the agent at the directory and it registers the model with the router: the model joins your project's model list, endpoints can serve it, and the weights never cross the network. The platform keeps no copy of the files.

Private inference agents only. Execution agents cannot hold models, so the permission below is refused for them.

1. Allow the Agent to Preseed

New agents are created with Allow model preseed already ticked; clear it on the create form if you would rather this agent never registered local weights. Agents created before this default shipped have it off. To change it on an enrolled agent, open its Actions menu, choose Edit, and set Allow model preseed there -- or use the API:

The CLI carries the same switch: erectl agents AGENT_ID --update --allow-preseed, and --no-allow-preseed to take it back.

The import is a local filesystem operation, so nothing checks the permission while it runs. The check happens when the agent registers the model with the router. Grant it first, or the registration is rejected and the model never appears.

2. Import the Model

The source must be a Hugging Face model directory: a config.json plus at least one .safetensors or .bin weight file. A source that is a symlink into a Hugging Face cache is resolved to the real files first.

Stop the agent before importing. The import takes exclusive access to the model cache and refuses to start while a running agent holds it.

The command prints the model id, the link mode it resolved, the file count, and the size. That id is the id the model carries in your project once it is registered.

Link Modes

The link mode decides how the files get into the cache, and what it costs you in disk.

Importing the same directory twice is a no-op. A source whose files already match a cache entry returns that entry and writes nothing.

3. Start the Agent

The agent registers its imported models when it starts and again on every reconnect. Once the router accepts one, it appears in the project's model list with a Preseeded badge; see Model Management for what you can do with it from there.

A rejection is logged with its reason: the agent is not allowed to preseed, the name is already taken by another model in the project, or the model was deleted from the project. In the delete case the agent purges its own copy; in the others, fix the cause and the next registration goes through.

Importing from the macOS App

The macOS application has an Import Model pane: pick the directory, pick the link mode, watch the checksum progress. The app owns its agent, so you do not stop anything first. With the agent running the model is registered immediately; with it stopped, at the next start. See EIM on macOS.

Importing in a Container

The import must run inside the agent's mount namespace, or it writes to a cache the agent never reads. It also cannot run while the agent holds that cache. So: stop the agent container, then run the import as a one-shot container carrying the same cache volume plus the source directory. The image entrypoint passes the arguments straight through.

Two constraints follow the mounts. Hardlinking needs the source and the cache on one filesystem, so a source bind-mounted from elsewhere lands on copy instead. And the files have to be readable by the UID the agent runs as.

Listing and Removing Imported Models

The agent can tell you what it holds and where each model stands with the router. Safe to run while the agent is running:

Each model reports registered (the router accepted it and it is in your model list), refused with the reason, or not announced when no verdict has arrived yet.

Removal has two halves, and they are not interchangeable. Deleting the model on the platform is the complete removal: the router tombstones it and asks every agent holding it to purge, and an agent that misses that command self-purges on its next announce. Do that from your model list, the desktop app's Import Model pane, or the API.

Removing the local copy deletes the files on one machine and nothing else -- the model stays in your project's list, pointing at bytes that are gone, until you delete it on the platform too. An agent holds no management credential, so this is all the CLI can do:

The agent must be stopped for remove-model (it holds the cache lock while running). Source files outside the model cache are never touched, so a hardlink or symlink import leaves your originals alone. Models the platform downloaded are refused: removing one only forces the next provision to download it again.

What a Preseeded Model Does Not Do

• It is not shared. The weights stay on the agents that imported them. The platform makes no copy, publishes nothing to the catalog, and refuses share and unshare.
• Nothing reads the files in the background. Revalidation and metadata refresh are refused, and the background sweeps skip preseeded models. The only reader is the engine serving it.
• It has no versions. Creating, promoting, and rolling back a version are refused.
• It runs only where its bytes are. Endpoints can only bind agents that hold the files. The endpoint wizard blocks the other pairings and names the holders; creating or attaching past it is refused.
• It uses no storage quota. Your quota counts bytes the platform stores, and it stores none of these.

Metadata is the exception: it stays yours to edit and it still drives what the agent launches the engine with. See Model Management.

Deleting a Preseeded Model

• Unbind it first. Delete is refused while an endpoint or an anchor endpoint binds the model.
• Deleting removes it everywhere. The model leaves your project and every agent holding the files purges its cached copy. An agent that was offline at the time purges on its next registration attempt.
• The purge stays inside the agent's model cache. Your source directory is untouched.
• Importing the same directory again creates a new model with a new id. Deleting bans the entry, not the files.

Agent Monitoring

The agent dashboard at /agents provides real-time visibility into your enrolled agents:

• Heartbeat status: whether each agent is actively communicating with the router.
• Current load: active request count and capacity.
• Model information: which model is loaded and its serving state.
• Event history: per-agent log of significant lifecycle events.

Programmatic Access

For programmatic snapshot access to your fleet, use the documented REST endpoint GET /:projectId/v1/management/agents (see Agent API) with a project API key.

Heartbeat Tracking

The router tracks the last_heartbeat_at timestamp for each agent. An agent with a heartbeat older than the staleness threshold (default 30 seconds; the deployment sets it) is considered disconnected and excluded from routing decisions until it reconnects and resumes sending heartbeats. The effective status shown in the dashboard and API is derived from both the stored status and heartbeat freshness.

Agent Controls

Each row's Actions menu carries three verbs beyond Edit and Delete. All three act on agents you own; none of them re-enrol the agent or touch its downloaded models.

Reset and Disable both ask for confirmation in the dashboard. How an agent comes back from a Reset depends on how it runs: a container with a restart policy or a systemd unit starts a fresh process; the macOS desktop app hosts the agent itself and cycles it in place, without closing the app; an agent run in the foreground stays stopped until you start it again.

Agent Lifecycle

Agents transition through the following status states. The status field in the database reflects the stored lifecycle state. The effective_status is computed at runtime from the stored status and heartbeat freshness.

Allowed Transitions

State Diagram

stateDiagram-v2
    [*] --> pending: join key issued
    pending --> active: enrollment handshake
    pending --> dead: join key expired
    active --> suspended: PATCH status=suspended
    active --> disconnected: heartbeat timeout
    active --> dead: extended absence
    disconnected --> active: heartbeat resumes
    disconnected --> suspended: owner suspends
    disconnected --> dead: extended absence
    suspended --> active: PATCH status=active
    suspended --> dead: extended absence
    dead --> [*]
                

Single-Agent Queuing

When only one agent is available for an endpoint's tier, the router queues incoming requests instead of immediately returning a 503 error. This matters for EIM deployments with a single backend agent.

The queue timeout is 30 seconds by default; the deployment sets it. If the agent does not become available within the timeout, the request fails with a timeout error.