EIM on macOS
On Apple Silicon, the Erebine Inference Microservice (EIM) runs inside a native macOS application. It installs, configures, and controls the EIM in one window, running Metal-accelerated vLLM or the Laya engine for System 1 decision models. Paste a join key and the Mac enrolls into the inference mesh as an EIM. No container, no admin password. Sign in and the same window becomes the console for the rest of the platform: 17 destinations in four groups, each one stating what it needs.
- platform
- Apple Silicon (arm64), macOS 26+
- accelerator
appleMetal(unified memory, TP 1)- runs as
- logged-in user, inside the app (no admin)
Why a Native App
vllm-metal needs native Metal access (MLX plus Metal kernels and an arm64 Python 3.12 runtime). Apple's container binary runs Linux guests with no GPU or Metal passthrough, so there is intentionally no container image for this backend. The macOS app is the installer, environment-prep, and control surface in one place, and everything runs as the logged-in user.
The CUDA, ROCm, and CPU paths still use the container stacks documented on the EIM Overview page. This page covers the Apple Silicon path only.
Requirements
- Apple Silicon Mac (arm64) running macOS 26 or newer.
- A join key from the dashboard: System -> EIMs and EEMs -> New EIM.
- Outbound HTTPS to the router URL (enrollment) and outbound TCP to the router's CurveZMQ port (data plane).
curl, uv, and Python 3.12 are provisioned automatically; uv and a pinned 3.12 are installed on first run if missing.
Install
Erebine Desktop is published with each release as Erebine-Desktop.dmg for Apple silicon; get it from the downloads page. What follows is the setup path inside the application.
- Open Erebine from Applications and accept the license agreement. On a Mac with no session and no EIM or EEM set up, the app then opens on a full-window sign-in. Choose Use this Mac without an account to go straight to the EIM, which the app remembers, or sign in. The EIM destination shows Setup until the EIM is installed.
- Confirm the host preflight passes, paste your join key, and click Install & Start.
Erebine Desktop tells you when a newer release exists; it never updates itself. On launch and once a day it asks GitHub for the latest release of Erebine/binaries. The request is anonymous and carries the app's version in its User-Agent (Erebine-Desktop/3.3.1), the Accept and Accept-Language: en headers, and the ETag of the previous answer. It carries no account, project, key, cookie or machine identifier; GitHub sees your IP address, as any site does. When a newer version is out, the sidebar footer and Settings show Update available with the version. Click it to read the release notes and choose Download, which opens Erebine-Desktop.dmg in your browser, or Later. Erebine -> Check for Updates... checks at once. Turn the automatic check off under Settings, Updates, Check for updates automatically.
The app reports Apple Silicon as a single appleMetal accelerator sized from the Metal unified-memory budget; tensor parallelism stays 1. Once enrollment succeeds, the Mac appears under System -> EIMs and EEMs with status online.
Inference Engine
The EIM runs one of two inference engines, vLLM or Laya. vLLM (vllm-metal) serves chat and the other vLLM task modes its packages support. Laya serves System 1 decision endpoints (System 1 Decisions) on the Mac's GPU through PyTorch's Metal backend, mps.
Pick the engine in onboarding, above Install & Start, or later on the Engine card in Settings. Each choice reads installed or not installed. In Settings, choosing one that is not installed and clicking Apply & Restart installs it first; the app saves the choice and restarts the EIM only after the install succeeds. While it installs, the EIM keeps running its current engine and Settings are locked. A failed install applies nothing: the EIM keeps running the engine it had, and a notice names the step that failed.
Switching can unbind endpoints. If this Mac is bound to an endpoint the new engine cannot serve, the router removes that binding when the EIM reconnects, and an endpoint left with no EIM reads EIM '<name>' no longer serves task mode '<mode>'. The picker warns whenever your choice differs from the engine the EIM runs (in onboarding, once this Mac is enrolled), whether or not such a binding exists, because a stopped EIM cannot report its bindings.
| Engine | Installs | Serves |
|---|---|---|
| vLLM | vLLM 0.33.0 (the prebuilt macOS wheel) and the vllm-metal 0.33.0 plugin into ~/.venv-vllm-metal, the HuggingFace compat shim, and the vLLM wrapper. A runtime installed by an earlier Erebine Desktop keeps its versions until Update runtime runs. |
Chat, and the other vLLM task modes its packages support. |
| Laya | laya 0.4.30 and PyTorch 2.14.1 with Metal into ~/.venv-laya, from a pinned, hashed lock, and the Laya launcher in ~/.local/bin. |
systemone |
Update runtime in Settings and Uninstall on the dashboard act on the engine the EIM runs, not on the picker's choice. Update runtime rebuilds that engine: for Laya, ~/.venv-laya from the pinned lock, with the launcher rendered again; for vLLM, vLLM from source with the latest Metal plugin. Uninstall stops the EIM, deletes this Mac's local enrollment, and removes that engine's launcher or wrapper. Uninstall & purge both engines also deletes both engine environments (~/.venv-vllm-metal and ~/.venv-laya), both launchers, the vendored libraries, and the saved settings.
With Laya selected, Settings hides the vLLM-only controls -- the memory allocation, sequences, context and batched-token limits, model format, request timeouts, speculative decoding, and the pass-through arguments and environment -- and the app passes none of their saved values to the EIM.
Sign In
Enrolling the EIM needs a join key. The two cloud contexts need an account, and the app signs in as a member rather than asking you to paste an API key. There is no key field, and no key to copy: key material never crosses the clipboard, a text field, or a screenshot.
A Mac with no session and no EIM or EEM set up signs in on a full-window screen that follows the license agreement. It names the site it signs in to, with Change for another site, and offers Create an account and Use this Mac without an account. Any other Mac opens into the app, and signs in from Project.
- Email and password. An account with two-factor enabled is then asked for a TOTP code or a backup code.
- GitHub. Opens a system web-authentication sheet with an ephemeral browser session, so the app does not inherit a browser login you did not choose to hand it.
- Apple. Runs in the same kind of browser sheet.
Both sign-ins, the first-launch screen and the sheet Sign in opens on Project, show GitHub and Apple only when your site offers them, and take the six-digit code, or a backup code, in the same form.
On success the app mints one API key for this Mac, named Erebine Desktop (<hostname>), expiring 90 days out. It always carries the inference scope, and adds execution and management when your role in the project grants them; a role that grants neither gets inference alone. It goes straight into the macOS Keychain and is never displayed. The hostname is in the name because the project's key list is where a lost laptop gets revoked, and identical rows tell you nothing. Revoke it from Integrations -> Keys or from the dashboard.
The app's router calls go on that key. Exec, Approvals and Learnings need execution; Maintenance, the Fleet write actions and a model's engine settings need management. A call the key's scopes do not cover is refused with a 403, and the surface says which scope it needed.
Signing out ends the member session and closes the cloud destinations. It does not stop the EIM. EIM, Import and Logs need no cloud credential, stay available, and the Mac keeps serving traffic. A session that lapses while the Mac is offline behaves the same way: the cloud destinations ask you to sign in again, and the EIM carries on.
What You Can Reach
Two different things can stand between you and a destination, and the app does not conflate them.
Not right now. A destination blocked by state stays in the sidebar and stays selectable. Opening it renders the reason as text on the surface -- "Sign in to Erebine to open this.", "Choose a project to open this." -- rather than as a tooltip on a greyed-out row. A disabled control is out of the macOS keyboard focus order, so a tooltip on one is unreachable without a mouse.
Not for this account. A destination the account cannot be granted is not listed. The sidebar footer then reads N not available, and opening it names every withheld destination beside the specific thing that would grant it. Withholding a destination silently would be indistinguishable, from the operator's side, from never having shipped it.
Withholding needs proof. The app hides a destination only when it can establish positively that the account cannot use it: today that is Exec, when this Mac's key does not carry the execution scope. Where the app cannot establish an entitlement it lists the destination and lets the server answer. A 403 you can read costs less than a destination you can never find.
Entitlements resolve when you choose a project, not continuously, so the sidebar does not rearrange itself while you are reading it.
Integrations is never hidden on a key scope. It holds Keys, Webhooks and MCP, and the three have different requirements -- MCP is local wiring and needs nothing at all. Hiding the destination on the strictest of them would take two reachable tabs down with it.
Control Surface
The Inference Microservice context in detail. These need no cloud credential and work signed out.
| Surface | What it does |
|---|---|
| EIM | Setup until the EIM is installed, then live status, the Apple Metal accelerator and unified-memory budget read from Metal, and Start / Stop / Restart / Uninstall. |
| Import | Import a model into the EIM's model cache -- a directory on this Mac, or a Hugging Face repository the app downloads itself -- list what has already been imported with each model's router status, optionally set its metadata and create an endpoint for it, and remove models you no longer want. Works while the EIM runs or while it is stopped -- a stopped EIM is started so the model reaches the router rather than sitting in the cache unseen. The Advanced group, collapsed by default, sets the description, license, Hugging Face repository, workload and capability tags, context length, parsers, trust remote code and HF config overrides at import. Once the router registers the model they are applied in one model update, and only then is the endpoint created, so its first launch already carries them. If the update is refused, no endpoint is created; both outcomes show in the pane and in Logs. Reasoning effort and speculative decoding are set from Models after import. The Hugging Face source takes a repository id or a pasted model page, a revision (main by default) and optional include and exclude globs. Before anything downloads it shows the size, the file count, whether the repository is gated and the free space, and disables the import with the reason when it cannot proceed. The download shows its bytes, files and rate with Cancel; a cancelled or interrupted download offers Resume, which keeps every finished file, and Discard, and so does a download whose registration failed, with Retry registration. The outcome stays on the pane when you switch away, and downloads left unfinished by an earlier run are listed when the app starts; one downloaded from a different Hub endpoint offers Discard alone. Retry registration asks the Hub for the revision again and downloads only files that are missing or changed. A token for gated and private repositories is checked with the Hub endpoint when you save it, stored in the macOS Keychain for that endpoint, and sent to it alone; pointing EREBINE_EIM_HF_ENDPOINT at another endpoint needs a token saved for that one. A mirror that cannot check tokens stores it unverified and says so; huggingface.co always checks. A gated repository's refusal links its model page and quotes Hugging Face's own message. The download follows the HTTPS proxy set in System Settings, Network; a proxy auto-configuration (PAC) file is not followed. See Preseeding a Local Model. |
| Settings | EIM configuration, split across two tabs. Essentials leads with this Mac's accelerator, memory budget and free memory, then the allocation the engine will take -- as a whole percent, with a recommendation derived from what is free right now. Connection, request limits and metrics sit beside it. Advanced holds engine tuning (sequences, context, batched tokens, load timeout, cache size, quantization, KV cache), speculative decoding, request timeouts, and the pass-through argument and environment escape hatches. Every field states what leaving it blank resolves to. Apply & Restart saves and restarts the EIM with them. |
| Logs | The EIM's stdout and stderr, tailed live. |
| Quantization on this platform | The macOS runtime is vllm-metal, whose kernels are MLX rather than CUDA. AWQ loads through an MLX repack and is limited to 4-bit, group size 128, with a zero point. GGUF and MLX checkpoints are detected when the model loads and need no setting at all. fp8, GPTQ and bitsandbytes have no Metal implementation, so the Settings pane does not offer them -- choosing one produced a failed model load, not a slower one. |
| Menu bar | Status at a glance with quick start and stop. |
The EIM runs inside the app, so there is no background service to inspect and no log file to tail: its log streams to Logs.
Model Settings
Selecting a model your project owns in Models opens its settings, in the sections the dashboard's model page uses: Listing, Capabilities, Reasoning effort, Engine, Dimensions, Weights, Speculative decoding and HF config overrides. Each section saves on its own and sends only the keys you changed, through the same model update the API documents. A model another project owns is read-only.
- Set and clear. A value you set is kept when the model is revalidated. Clear sends
null: a value read from the model's files comes back on the next revalidation, and a parser returns toauto. A model imported from a Mac is never revalidated, so a field you clear on it stays empty. - Engine settings. When a save changes an engine setting, the section names it beside Save: the save restarts the engines on the model's endpoints and needs an API key with the
managementscope. Without the scope nothing is stored, and the section says so. - Capabilities are saved as the whole list, with where the list came from:
inferred,probedoroverridden.structured_outputis read-only; only the capability probe sets it. - HF config overrides are replaced as a whole set. Preview JSON assembles the rows without storing them.
- Trust remote code is the project owner's to change.
A value the router refuses is shown beside its field, and keyboard focus moves to it.
What Install Does
Behind the Install & Start button the app, in order:
- Preflights the host (arm64, macOS,
curl,uv). - Provisions Python 3.12 via
uv. - Copies the runtime libraries the app bundles into place.
- Installs the selected engine: for vLLM, vllm-metal into
~/.venv-vllm-metaland the HuggingFace compat shim; for Laya, laya and PyTorch into~/.venv-laya. See Inference Engine. - Renders the engine's wrapper or launcher.
- Enrolls with the join key and starts the EIM inside the app.