EIM on Windows
On Windows, the Erebine Inference Microservice (EIM) runs in a container inside a WSL2 distribution. WSL2 has no Linux GPU driver: every GPU is WSL's /dev/dxg device, backed by the Windows driver. NVIDIA and AMD GPUs run there; erectl deploy eim detects WSL and deploys the shape each vendor needs.
- platform
- Windows 11, a WSL2 distribution with systemd
- GPU device
/dev/dxg(no/dev/kfd, no/dev/nvidia*)- runs while
- the WSL distribution runs
What Runs
| Hardware | Under WSL2 | How |
|---|---|---|
nvidia-gpu | Runs | The NVIDIA container toolkit's WSL mode hands the container /dev/dxg and the Windows driver's libraries as one device, nvidia.com/gpu=all. More than one EIM shares the GPUs; MIG and one GPU per EIM are not available. |
amd-gpu | Runs, one GPU | /dev/dxg, Microsoft's libdxcore.so mounted read-only, unprivileged, PyTorch's expandable segments off. The ROCm in eim-vllm-rocm carries AMD's WSL bridge library. |
intel-gpu | Refused | No WSL2 path for Intel GPUs. Run the Intel EIM on native Linux. |
cpu, amd-cpu | Runs | As on native Linux. |
eim-vllm-rocm carries kernels for gfx1200 and gfx1201 (RDNA 4) and gfx1150 to gfx1153 (Ryzen AI APUs). AMD also supports RDNA 3 GPUs (gfx1100 to gfx1102) under WSL2; the image has no kernels for them.
On Windows
- Windows 11 and a current WSL: run
wsl --update. The distribution must run under WSL2 (wsl --set-version <distribution> 2). - The current GPU driver from the GPU vendor. For AMD, AMD Software: Adrenalin Edition or PRO Edition.
- The WSL VM's memory and CPUs are the EIM's. Set them in
%UserProfile%\.wslconfig, under[wsl2]:memory=(half of Windows' memory by default) andprocessors=. - WSL stops a distribution that has been idle for
instanceIdleTimeout, 15 seconds by default. To keep the EIM running when no terminal is open, set it under[general]:
[general]
instanceIdleTimeout=-1
Run wsl --shutdown after changing .wslconfig. Windows does not start a distribution at boot; the EIM starts when the distribution does.
In the Distribution
- systemd, for
--service. Add to/etc/wsl.conf, then runwsl --shutdownon Windows:
[boot]
systemd=true
- podman with podman-compose, or docker with the compose plugin.
- NVIDIA: the NVIDIA container toolkit, which generates its WSL-mode device spec itself.
- AMD:
/dev/dxgand/usr/lib/wsl/lib/libdxcore.so, which WSL provides. No ROCm is needed in the distribution. With ROCm installed there,--auto-hardware-detectfinds the GPU through itsamd-smi; without it, pass--hardware-type amd-gpu.
Deploy with erectl
sudo erectl deploy eim --hardware-type amd-gpu --service --region us-east
The plan, printed before every deploy and by --dry-run, says the host is WSL2 and what that changes. On WSL2 the command:
- refuses, before it changes anything, an AMD GPU without
/dev/dxg, without/usr/lib/wsl/lib/libdxcore.so, or on a kernel that is not a WSL2 kernel, naming what is missing and what to do; - refuses
--servicewhen systemd is not running, with the/etc/wsl.conflines; - refuses more than one AMD EIM,
--gpu-linkand--no-sysctlfor an AMD GPU,--migand a pinned layout of more than one NVIDIA EIM, and--hardware-type intel-gpu; - installs a unit that waits for
/dev/dxg(and, for AMD,libdxcore.so) before it starts the EIM.
See erectl deploy for every flag.
Deploy with Compose
From the erebine/container-agents repository, inside the distribution:
git clone https://github.com/erebine/container-agents.git
cd container-agents/compose
sudo mkdir -p /data/erebine/models /data/erebine/config /data/erebine/vllm-cache
sudo chown -R 5152:5152 /data/erebine
export EREBINE_EIM_JOIN_KEY=ejk_your_key_here
sudo -E podman compose -f compose.agent-amd-rocm-wsl.yaml up -d
compose.agent-amd-rocm-wsl.yaml is the AMD stack for WSL2. Its header comments explain each line that differs from compose.agent-amd-rocm.yaml, which does not run under WSL2. An NVIDIA GPU uses compose.agent-nvidia.yaml unchanged.
What Differs from Native Linux
- GPU memory is shared with Windows. The desktop holds part of it, and its use moves. The EIM sizes the engine's GPU memory from what is free when a model launches, at most 85% of the GPU.
EREBINE_EIM_WSL_HEADROOM=offturns that off. - Readings. On AMD, amd-smi reports the GPU's model, architecture, memory, activity, power and temperature, and no PCIe link, firmware, NUMA node or RAS figures; per-process memory is not reported. The EIM needs none of the missing ones and binds no NUMA node.
- Allocator. On AMD, PyTorch's expandable segments are off, so GPU memory can fragment more under long, mixed load.
- Lifetime. The EIM runs while the WSL distribution runs, not from Windows boot. See On Windows for
instanceIdleTimeout. - Host settings. Kernel command-line changes go in
kernelCommandLineunder[wsl2]in.wslconfig;erectlprints the change and never writes that file.
Troubleshooting
erebine-agents: /dev/dxg did not appear within 120 sinjournalctl -u erebine-agents: the distribution has no GPU device. Runwsl --update, install the current GPU driver, runwsl --shutdown. Checkls -l /dev/dxg.erebine-agents: /usr/lib/wsl/lib/libdxcore.so did not appear within 120 s: WSL has not mounted the Windows GPU libraries. The same steps; checkls /usr/lib/wsl/lib.erebine-agents: /dev/kfd did not appear within 120 son WSL2: the unit was written for native Linux, by an erectl that did not detect WSL. Run the deploy again with the current erectl.[DxcoreLoader] Cannot load libdxcore.so, orhsa_initfailing, in the EIM's log: the container lackslibdxcore.soon its loader path. Mount/usr/lib/wsl/lib/libdxcore.soat/usr/lib64/libdxcore.so; mounting/usr/lib/wslalone is not enough.CUDA error: invalid argumenton the first GPU allocation of an AMD EIM: PyTorch's expandable segments are on. SetPYTORCH_ALLOC_CONF=expandable_segments:False.--service on WSL: this distribution runs without systemd: turn systemd on in/etc/wsl.conf(see In the Distribution).GPU image found no GPU it can drive: the container was started without/dev/dxg, or for AMD without thelibdxcore.somount. The log line names what detection found.