Qwen3.8-27B-DFlash2

Engine
vLLM
Listed
1 Oct 2026

This repository contains the DFlash 2 draft model for Qwen/Qwen3.8-27B. It is not a standalone language model: it runs inside a speculative decoding server and drafts tokens for the target model to verify. The checkpoint is also mirrored at z-lab/Qwen3.8-27B-DFlash2.

Serves
Chat, Reasoning
Parameters
4.2B
Context
262,144 tokens
Weights
3.8 GB
License
Apache-2.0

Capabilities

Read from the model's capability tags. Requests are admitted per tag.

Chat chat
  • POST /v1/chat/completions
  • POST /v1/responses

Endpoint task mode generate

Inputs
Text
Reasoning effort reasoning_effort
none, low, medium, high
Values a request may send; the model family sets the vocabulary.
Structured output structured_output
not yet probed
Admitted until the capability probe observes the model.
Forced tool choice forced_tool_choice
not yet probed
Admitted until the capability probe observes the model.

Launch settings

Every engine serving the model starts with these.

Set by the owner. auto: the serving node derives it. default: the platform's value, left in place.

Engine
vLLM
Context window --max-model-len
262,144 tokens
Quantization --quantization
none default
Weights load as stored, bfloat16.
KV cache dtype --kv-cache-dtype
auto
No flag: the checkpoint's KV quantization, else the model dtype.
Scheduling policy --scheduling-policy
priority default
Interactive requests ahead of batch work; same-class traffic in arrival order.
Tool-call parser --tool-call-parser
auto
Resolves to qwen3_coder from the architecture.
Reasoning parser --reasoning-parser
auto
Resolves to qwen3 from the architecture.
Trust remote code --trust-remote-code
on default
Python shipped in the bundle may run at load (the default).

Engine tuning

Every setting auto: the serving node derives each one.

Set by the owner. auto: the serving node derives it.

Mixture of experts

Expert parallel --enable-expert-parallel
auto
Expert load balancing --enable-eplb
auto
All-to-all backend --all2all-backend
auto

Memory and batching

GPU memory utilization --gpu-memory-utilization
auto
Max sequences --max-num-seqs
auto
Max batched tokens --max-num-batched-tokens
auto
Long-prefill threshold --long-prefill-token-threshold
auto

Execution

Enforce eager --enforce-eager
auto
Prefix caching --enable-prefix-caching
auto
Async scheduling --no-async-scheduling
auto
Custom all-reduce --disable-custom-all-reduce
auto

Speculative decoding

Draft tokens checked by the model in one pass.

Speculative decoding
not configured
An endpoint's own setting, or the serving node's default for the model family, applies.
Drafts for
Qwen3.8-27B-FP8
This model is the speculative draft of a listed model.
Drafts for
Qwen3.8-27B
This model is the speculative draft of a listed model.

Architecture

Read from the model's files at upload.

Architecture
qwen3
Parameters
4,194,355,200
Layers num_hidden_layers
5
Hidden size hidden_size
5,120
Attention heads num_attention_heads
32 (8 KV)
Head dim head_dim
128
Vocabulary vocab_size
248,320
Native dtype torch_dtype
bfloat16
Weights on disk
3.8 GB, safetensors

Call it

Create an endpoint for the model, then send it a request in the OpenAI shape.

Shell
# Create an endpoint for this model, then provision it. # --tier-id is the service tier to run on; pick one at /endpoints/new. erectl endpoints create \ --name "My endpoint" \ --slug my-endpoint \ --model-id 496B125F-7AEE-41FF-851B-DE96CB42D7E5 \ --tier-id <tier-id> # provision takes the endpoint UUID returned by create erectl endpoints provision <endpoint-id> # Call it. Requests are routed by /<project-id>/<endpoint-slug>. curl https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $EREBINE_API_KEY" \ -d '{ "model": "Qwen3.8-27B-DFlash2", "messages": [{"role": "user", "content": "Hello, world!"}], "stream": true }'

Model card

Images are left out; its HTML tables are set as tables.

Blog | GitHub

DFlash 2 is a block-diffusion drafter for speculative decoding. It predicts
a whole block of tokens in a single pass and keeps the top candidates at
every position. A lightweight selector then traces one coherent path through them.
Two-tap dynamic convolutions in the backbone keep the draft from decaying
toward the end of the block. Decoding is lossless: greedy output
matches the target model exactly, and sampling preserves its distribution.

Quick Start

Serve with SGLang:

pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"

python -m sglang.launch_server \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
  --speculative-num-draft-tokens 8

Or with vLLM:

pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head"

vllm serve Qwen/Qwen3.8-27B \
  --speculative-config '{
    "method": "dflash",
    "model": "incoai/Qwen3.8-27B-DFlash2",
    "num_speculative_tokens": 7
  }'

See the blog post for other engines and more details.

Evaluation

  • Runtime: SGLang on one NVIDIA H200, with FlashAttention 3 for target and draft attention
  • Speculation block size: 8 (7 draft tokens per verification step)
  • Sampling: Qwen3.8's officially recommended parameters (temperature 1.0, top-p 0.95, top-k 20), with xhigh reasoning effort
  • Maximum new tokens: 4096
  • Prompts: benchmark formatting from z-lab/dflash

We compare autoregressive decoding, Qwen3.8's built-in seven-token MTP,
a community DSpark drafter
(RadixArk/Qwen3.8-27B-DSpark),
and DFlash 2. All speculative methods propose seven draft tokens per
verification step.

Acceptance Length

Acceptance length is the per-request mean of completion tokens divided by verification steps.
Higher is better.

TaskMTPDSparkDFlash 2
GSM8K5.024.365.46
MATH-5004.723.925.28
HumanEval3.913.304.39
MBPP3.993.514.79
MT-Bench3.743.014.10

Throughput

Throughput is total output tokens divided by end-to-end wall time.
Each cell shows output tok/s (speedup vs. autoregressive).

Concurrency 1
TaskAutoregressiveMTPDSparkDFlash 2
GSM8K68.9178.5 (2.59×)185.3 (2.69×)236.1 (3.43×)
MATH-50069.0172.8 (2.51×)174.5 (2.53×)230.7 (3.34×)
HumanEval69.0151.9 (2.20×)159.9 (2.32×)214.6 (3.11×)
MBPP69.0153.1 (2.22×)163.3 (2.37×)226.9 (3.29×)
MT-Bench68.9134.9 (1.96×)137.6 (2.00×)184.0 (2.67×)
Concurrency 8
TaskAutoregressiveMTPDSparkDFlash 2
GSM8K467.21,022.1 (2.19×)1,040.8 (2.23×)1,328.7 (2.84×)
MATH-500480.01,023.5 (2.13×)1,025.8 (2.14×)1,368.3 (2.85×)
HumanEval483.4934.2 (1.93×)956.5 (1.98×)1,291.5 (2.67×)
MBPP478.0938.1 (1.96×)974.1 (2.04×)1,328.0 (2.78×)
MT-Bench480.5835.2 (1.74×)802.3 (1.67×)1,090.2 (2.27×)
Concurrency 32
TaskAutoregressiveMTPDSparkDFlash 2
GSM8K1,329.81,381.1 (1.04×)1,506.5 (1.13×)1,922.5 (1.45×)
MATH-5001,505.81,415.6 (0.94×)1,429.0 (0.95×)1,951.8 (1.30×)
HumanEval1,546.51,296.8 (0.84×)1,330.1 (0.86×)1,799.0 (1.16×)
MBPP1,507.71,314.9 (0.87×)1,361.3 (0.90×)1,886.8 (1.25×)
MT-Bench1,507.41,159.7 (0.77×)1,115.5 (0.74×)1,525.3 (1.01×)

Citation

If you find DFlash 2 useful, please cite:

@misc{inco2026dflash2,
  title  = {{DFlash 2: Keep Drafting Parallel}},
  author = {{Inco AI}},
  year   = {2026},
  month  = {August},
  url    = {https://inco.ai/blog/dflash2/}
}

Please also cite the original DFlash paper:

@inproceedings{chen2026dflash,
  title     = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author    = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2026}
}