Service Tiers
Seven tiers, chosen per endpoint at creation. Free for evals, pay-per-token on shared CPU and GPU pools, or flat-hourly on your own workers. A tier prices the request; hardware is a separate, per-endpoint requirement. The request-level service_tier hint only nudges priority and billing within the tier you already picked.
A tier is a pricing plane. It sets the per-token or hourly rate, rate limits, timeouts, concurrency, and scheduling priority. It does not constrain hardware: hardware eligibility is declared per endpoint as a hardware requirement and evaluated against each worker's reported capability at routing time. See Hardware Requirements.
Tiers are divided into three categories:
- Free, For evaluation and testing with shared CPU resources.
- Compute (CPU and GPU Shared), Pay-per-token tiers with shared accelerator pools.
- Self-Hosted, Bring your own infrastructure with flat hourly billing and no token metering.
User-selectable tier slugs (identifiers used in the API and configuration) are:
free, cpu_amd_optimized, cpu_intel_optimized,
gpu_nvidia_shared, gpu_amd_shared, gpu_intel_shared,
self_hosted.
Dedicated GPU tiers are deprecated. The legacy
gpu_nvidia_dedicated and gpu_amd_dedicated slugs are
retained for backwards compatibility only. New endpoints created against these
slugs are transparently remapped to their shared equivalents
(gpu_nvidia_shared and gpu_amd_shared). Do not select
a dedicated tier when creating new endpoints; pick the shared variant
directly. Pinned, single-tenant GPU capacity is not currently offered as a
self-service tier.
Figures are behind sign-in. Per-token, cached, and hourly rates are shown to signed-in accounts. Everything else on this page -- what each tier bills for, the multipliers, the limits, and the caps -- is the same either way. For a quote without an account, mail contact@erebine.ai.
Tier Comparison
Pricing and Rate Limits
| Tier (slug) | Pricing | Tokens/Min | Requests/Min | Max Model Size |
|---|---|---|---|---|
Free (free) |
No charge | 10,000 | 64 | 48 GB |
CPU AMD Optimized (cpu_amd_optimized) |
Per token | 100,000 | 128 | Unlimited |
CPU Intel Optimized (cpu_intel_optimized) |
Per token, same rate as CPU AMD | 100,000 | 128 | Unlimited |
GPU NVIDIA Shared (gpu_nvidia_shared) |
Per token, highest shared-GPU rate | 500,000 | 256 | Unlimited |
GPU AMD Shared (gpu_amd_shared) |
Per token, lowest shared-GPU rate | 500,000 | 256 | Unlimited |
GPU Intel Shared (gpu_intel_shared) |
Per token, tied with GPU AMD | 500,000 | 256 | Unlimited |
Self-Hosted (self_hosted) |
Flat hourly, no token meter | Unlimited | Unlimited | 512 GB |
"Unlimited" in the rate limit columns means no per-minute token or request cap is enforced. Model size limit of "Unlimited" in the database means no size check is applied -- the limit is determined by available VRAM on the assigned workers.
Cached tokens are charged at 25% of the full per-token rate on all billable tiers. See Cached Token Pricing for details.
Timeouts and Concurrency
| Tier | Request Timeout | Idle Timeout | Max Concurrent | Batch Size |
|---|---|---|---|---|
| Free | 30s | 120s | 8 | 1 |
| CPU AMD / Intel | 300s | 600s | 24 | 8 |
| GPU NVIDIA / AMD / Intel Shared | 300s | 600s | 48 | 16 |
| Self-Hosted | 1800s | 3600s | 96 | 64 |
All tiers except Free support both streaming responses and request batching. The Free tier supports streaming but not batching.
Tier Details
"Pool hardware" describes what the platform stocks each pool with. It is descriptive, not a gate: to require specific silicon for an endpoint, declare a hardware requirement on the endpoint.
Free (free)
Shared CPU pool. Two endpoints, two models, and a 1,000-credit grant per cycle to your first project while it has no payment method on file. The right tier for evaluating the routing surface and proving an SDK call works.
- Pool hardware
- Shared CPU pool
- Routing priority
- 10 (lowest)
- Credit grant
- 1,000 credits per cycle
- Endpoint cap
- 2
- Model cap
- 2 (<= 48 GB each)
CPU AMD Optimized (cpu_amd_optimized)
ZenDNN on AMD EPYC. Pick this for batch workloads where accelerator cost matters more than tail latency.
- Pool hardware
- AMD ZenDNN CPU
- Routing priority
- 20
- Distinguishing trait
- Same rate caps as Intel CPU; pick by available capacity.
CPU Intel Optimized (cpu_intel_optimized)
Intel oneAPI on Xeon. Same caps, concurrency, and priority as the AMD CPU tier; choose based on which silicon your workload benchmarks against.
- Pool hardware
- Intel oneAPI CPU
- Routing priority
- 20
GPU NVIDIA Shared (gpu_nvidia_shared)
NVIDIA CUDA on the shared GPU pool. The widest model compatibility of the shared GPU tiers because most published kernels target CUDA first.
- Pool hardware
- NVIDIA CUDA
- Routing priority
- 25
- Per-token rate
- Highest of the shared GPU tiers; widest kernel coverage
GPU AMD Shared (gpu_amd_shared)
AMD ROCm on the shared GPU pool. Same rate caps and scheduling priority as the NVIDIA shared tier, at a lower per-token rate.
- Pool hardware
- AMD ROCm
- Routing priority
- 25
- Per-token rate
- Lowest of the shared GPU tiers, tied with Intel
GPU Intel Shared (gpu_intel_shared)
Intel oneAPI on Arc and Max GPUs. Tied with AMD ROCm for the lowest per-token rate among shared GPU tiers.
- Pool hardware
- Intel oneAPI GPU
- Routing priority
- 25
- Per-token rate
- Lowest of the shared GPU tiers, tied with AMD
Self-Hosted (self_hosted)
Your workers, your infrastructure. No token metering, no request-rate cap, 1800s request timeout. Billed as a flat hourly rate for the routing and management layer, charged per enrolled worker for the hours it is enrolled. Workers are pinned to the project; model size is capped at a declared 512 GB.
- Pool hardware
- Your own workers, any vendor or form factor
- Routing priority
- 30 (highest)
- Max declared model size
- 512 GB
- Request timeout
- 1800s
- Billing
- Flat hourly; no token meter
Note on hardware matching. Like every tier, Self-Hosted carries no hardware constraint of its own. Self-hosted endpoints are bound to specific worker agents, and the router filters those candidates by the endpoint's declared hardware requirement and the model's VRAM footprint. Size self-hosted clusters for the workers you enrol and the requirements your endpoints declare.
Tier Selection
The service tier is configured per endpoint in the Erebine dashboard or API. When you create or update an endpoint, you select which tier it uses. All requests to that endpoint are routed to workers serving the configured tier that also satisfy the endpoint's hardware requirement.
The endpoint's configured tier always determines which worker pool is eligible
to serve a request; it cannot be overridden per request. The OpenAI-compatible
service_tier request parameter does, however, influence routing
within the configured tier and billing:
service_tier value |
Routing score adjustment | Token billing |
|---|---|---|
flex |
-15 (de-prioritized against other in-flight work) | Standard per-token rate |
default (or omitted) |
No adjustment | Standard per-token rate |
priority |
+15 (preferred against other in-flight work) | 1.25x the standard per-token rate |
The actual tier used is returned in the service_tier field of every
response (both streaming chunks and non-streaming completions). When a request
uses priority, both prompt and completion tokens are billed at the
1.25x multiplier on top of the endpoint tier's per-token rate; cached tokens are
still discounted by the tier's cached token multiplier before the priority
multiplier is applied.
The router selects backends within the configured tier using configurable routing strategies:
- Least Loaded, Prefer workers with the lowest queue depth.
- Lowest Latency, Prefer workers with the lowest predicted latency.
- Model Affinity, Prefer workers that already have the model loaded.
- Round Robin, Cycle through available workers evenly.
- Composite, Weighted combination of multiple strategies.
You can also pass optional X-SLO-TTFT-Ms and X-SLO-TPOT-Ms
request headers to hint at latency targets. The router boosts preference for workers
likely to meet these targets. See the
SLO Tracking documentation for details.
Hardware Requirements
Hardware constraints live on the endpoint, not the tier. An endpoint can declare a hardware requirement; the router routes only to workers that satisfy every field set:
- Vendor class, The accelerator family the model targets: NVIDIA CUDA, AMD ROCm, Intel oneAPI, Apple Metal, or CPU.
- Minimum VRAM per GPU, Each GPU on the worker must have at least this much memory. On unified-memory hardware such as Apple silicon, shared system memory counts toward the floor.
- Minimum GPU count, The worker must expose at least this many GPUs.
- Architecture class, A minimum GPU architecture generation, for models that depend on generation-specific kernels.
- GPU model pin, An optional exact GPU model match, for workloads validated on specific silicon.
- NVLink, Require an NVLink interconnect between GPUs, for multi-GPU workloads sensitive to interconnect bandwidth.
Unset fields do not constrain: an endpoint with no requirement runs on any worker serving its tier. Requirements are evaluated against the capability each worker reports and refreshes while connected, so a fleet change takes effect without redeploying the endpoint.
macOS Integration
Apple Silicon Macs (arm64, macOS 15+) integrate natively via the Erebine desktop application. The app installs, configures, and controls an inference agent with Metal-accelerated vLLM in a single window -- no container or admin password required. Once enrolled with a join key, the Mac appears under Infrastructure → Agents and serves as a shared-workload worker for the project.
The native path uses Apple Metal (not CUDA/ROCm) and declares
appleMetal as its accelerator class. Unified memory
is shared between CPU and GPU; the router budgets it from the
hardware requirement you
declare on the endpoint. See
EIM on macOS for the full deployment
guide.
Cached Token Pricing
When prefix caching is active and input tokens are served from the KV cache
(reported as prompt_tokens_details.cached_tokens in the usage
object), those tokens are billed at a reduced rate. The
cached_token_cost_multiplier for each tier determines the
fraction of the full per-token price charged for cached tokens.
| Tier | Cached Token Multiplier |
|---|---|
| Free | 0.0 (not metered) |
| CPU AMD / Intel Optimized | 0.25 (75% discount) |
| GPU NVIDIA Shared | 0.25 (75% discount) |
| GPU AMD / Intel Shared | 0.25 (75% discount) |
| Self-Hosted | 0.0 (not metered) |
The multiplier is the whole mechanism: it is the fraction of that tier's full per-token price charged for a cached token. Sign in for the two rate columns.
Maximizing your prefix cache hit rate directly reduces your token costs. See Prefix Caching for prompt structuring recommendations.
Custom Models and Tier Restrictions
Custom models, models that you upload and manage privately within your project --
are restricted to the Self-Hosted tier. To use a model on shared
compute tiers (Free, CPU, or GPU Shared), the model must be published to the public catalog
and promoted to the shared catalog role.
This restriction ensures that shared infrastructure only runs models that have been explicitly published and vetted. Private models never run on shared agents.
| Model Type | Allowed Tiers |
|---|---|
| Custom (private, not in catalog) | Self-Hosted only |
| Catalog model (deployable role) | Self-Hosted only |
| Catalog model (shared role) | All tiers |
Catalog Roles and Tier Access
Models in the public catalog are assigned a catalog role that controls which tiers they can be deployed on:
- Deployable: The default role for newly shared models. The model is visible in the catalog but can only be used with the Self-Hosted tier. Other users can see the model in the catalog but must deploy it on their own infrastructure.
- Shared: The model is available on all tiers, including shared compute.
The catalog role is managed automatically by the platform based on the state of the shared-agent model cache. There is no manual promotion endpoint and no operator workflow to flip the role directly:
-
When at least one shared agent has the model cached and ready to serve, the
platform promotes the catalog role to
shared. -
When no shared agent has the model cached, the platform demotes the catalog
role back to
deployable.
Because promotion follows shared-agent cache state, the set of catalog models available on shared tiers can change over time as agents load, evict, or rotate models.
For details on how model sharing works and how to publish models to the catalog, see the Model Sharing documentation.
Completion Storage
When store: true is set in a request, the completion is saved for
later retrieval. Completions pass through two storage stages before expiration:
When store: true is set in a request, the completion is
saved. A hot tier cache (Redis) fronts recently used content for fast
retrieval; older content is archived to object storage. Content is
retained as long as the project is funded - there is no
per-tier time-to-live. If the credit balance reaches zero, storage enters
a 96-hour grace window; settling the balance within it
removes nothing. See Pricing for the storage rate
and the published credit exchange.
Free Account Limits
Free plans grant 1,000 credits per cycle to your first project (while it has no payment method and no active subscription). When the credit balance reaches zero, token requests halt; storage keeps its 96-hour grace window before removal. There is no per-month token cap and no system-level storage quota - the credit balance is the only governor.
Choosing a Tier
| Use Case | Recommended Tier |
|---|---|
| Testing and prototyping | Free |
| Batch jobs where cost matters more than tail latency | CPU AMD or CPU Intel Optimized |
| Production traffic on a CUDA-targeting model | GPU NVIDIA Shared |
| GPU traffic where the lower per-token rate beats CUDA kernel coverage | GPU AMD or GPU Intel Shared |
| Models larger than 48 GB | Any paid Compute tier, or Self-Hosted up to 512 GB |
| Data sovereignty / compliance | Self-Hosted |
| Requests longer than 5 minutes | Self-Hosted (1800s timeout) |
When a model needs specific silicon, declare it in the endpoint's hardware requirement; the tier choice stays a price choice.
Change a tier from the endpoint settings page; new requests pick up the change on the next dispatch.
Code Examples
The endpoint's tier is fixed at creation; these requests do
not select one. Pass an optional service_tier
body field (flex, default, or
priority) to nudge routing priority and billing
inside the configured tier
(see Tier Selection). The
response echoes the actual tier used.
Python (OpenAI SDK)
from openai import OpenAI
client = OpenAI(
base_url="https://api.erebine.ai/proj_ABC123/my-endpoint/v1",
api_key="ere_my-project_abc123"
)
response = client.chat.completions.create(
model="my-model",
messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)
# The service_tier field shows which tier processed the request
print(f"Service tier: {response.service_tier}")
Node.js (OpenAI SDK)
import OpenAI from 'openai';
const client = new OpenAI({
baseURL: 'https://api.erebine.ai/proj_ABC123/my-endpoint/v1',
apiKey: 'ere_my-project_abc123'
});
const response = await client.chat.completions.create({
model: 'my-model',
messages: [{ role: 'user', content: 'Hello' }]
});
console.log(response.choices[0].message.content);
// The service_tier field shows which tier processed the request
console.log(`Service tier: ${response.service_tier}`);
curl
curl https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions \
-H "Authorization: Bearer ere_my-project_abc123" \
-H "Content-Type: application/json" \
-d '{
"model": "my-model",
"messages": [{"role": "user", "content": "Hello"}]
}'
# The response includes "service_tier" indicating the endpoint's configured tier
