DeepSeek-V4-Flash-0731
- Engine
- vLLM
- Listed
- 3 Nov 2026
This document describes the prompt encoding format used by DeepSeek-V4 series models. The encoding handles multi-turn conversations, tool calling, extended thinking (reasoning), and quick instruction tasks.
- Serves
- Chat
- Parameters
- 83.4B
- Context
- 1,048,576 tokens
- Weights
- 167 GB
- License
- MIT
Capabilities
Read from the model's capability tags. Requests are admitted per tag.
- Chat
chat -
POST /v1/chat/completionsPOST /v1/responses
Endpoint task mode
generate
- Inputs
- Text
- Reasoning effort
reasoning_effort - does not reason
- Structured output
structured_output - supported
- Proven by probe: a JSON schema request returned a payload that parses.
- Forced tool choice
forced_tool_choice - supported
- A forced tool_choice returns a call to that tool.
Launch settings
Every engine serving the model starts with these.
Set by the owner. auto: the serving node derives it. default: the platform's value, left in place.
- Engine
- vLLM
- Context window
--max-model-len - 1,048,576 tokens
- Quantization
--quantization - none default
- Weights load as stored, fp8.
- KV cache dtype
--kv-cache-dtype - auto
- No flag: the checkpoint's KV quantization, else the model dtype.
- Scheduling policy
--scheduling-policy - priority default
- Interactive requests ahead of batch work; same-class traffic in arrival order.
- Tool-call parser
--tool-call-parser - auto
- No parser for this architecture.
- Reasoning parser
--reasoning-parser - auto
- No parser for this architecture.
- Sampling defaults
generation_config.json - temperature 1, top_p 1
- Used when a request leaves them out.
- Trust remote code
--trust-remote-code - on default
- Python shipped in the bundle may run at load (the default).
Engine tuning
Every setting auto: the serving node derives each one.
Set by the owner. auto: the serving node derives it.
Mixture of experts
- Expert parallel
--enable-expert-parallel - auto
- Expert load balancing
--enable-eplb - auto
- All-to-all backend
--all2all-backend - auto
Memory and batching
- GPU memory utilization
--gpu-memory-utilization - auto
- Max sequences
--max-num-seqs - auto
- Max batched tokens
--max-num-batched-tokens - auto
- Long-prefill threshold
--long-prefill-token-threshold - auto
Execution
- Enforce eager
--enforce-eager - auto
- Prefix caching
--enable-prefix-caching - auto
- Async scheduling
--no-async-scheduling - auto
- Custom all-reduce
--disable-custom-all-reduce - auto
Speculative decoding
Draft tokens checked by the model in one pass.
- Speculative decoding
- not configured
- An endpoint's own setting, or the serving node's default for the model family, applies.
Architecture
Read from the model's files at upload.
- Parameters
- 83,439,268,220
- Layers
num_hidden_layers - 43
- Attention heads
num_attention_heads - 64
- Head dim
head_dim - 512
- Experts
- 256
- Vocabulary
vocab_size - 129,280
- Native dtype
torch_dtype - fp8
- Weights on disk
- 167 GB, safetensors
Call it
Create an endpoint for the model, then send it a request in the OpenAI shape.
# Create an endpoint for this model, then provision it.
# --tier-id is the service tier to run on; pick one at /endpoints/new.
erectl endpoints create \
--name "My endpoint" \
--slug my-endpoint \
--model-id 9E650D01-355E-4E8F-8CBB-35D09D69D2F7 \
--tier-id <tier-id>
# provision takes the endpoint UUID returned by create
erectl endpoints provision <endpoint-id>
# Call it. Requests are routed by /<project-id>/<endpoint-slug>.
curl https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $EREBINE_API_KEY" \
-d '{
"model": "DeepSeek-V4-Flash-0731",
"messages": [{"role": "user", "content": "Hello, world!"}],
"stream": true
}'Model card
Images are left out; its HTML tables are set as tables.
DeepSeek-V4 Encoding
A self-contained reference implementation is provided in encoding_dsv4.py.
Quick Start
from encoding_dsv4 import encode_messages, parse_message_from_completion_text
# Encode a conversation
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is 2+2?"},
]
prompt = encode_messages(messages, thinking_mode="thinking")
# => "You are a helpful assistant.What is 2+2?"
# Parse model output back to structured message
completion = "Simple arithmetic.2 + 2 = 4."
parsed = parse_message_from_completion_text(completion, thinking_mode="thinking")
# => {"role": "assistant", "reasoning_content": "Simple arithmetic.", "content": "2 + 2 = 4.", "tool_calls": []}Note: The parse_message_from_completion_text function is designed to handle well-formatted model output only. It does not attempt to correct or recover from malformed output that the model might occasionally generate. For production use, additional error handling is recommended.
Message Format
Special Tokens
| Token | Purpose |
|---|---|
INLINECODE2zE74A5889 | End of assistant turn (EOS) |
INLINECODE4zACED29E5 | Assistant turn prefix |
INLINECODE6zEF441620 / INLINECODE8zAD297779|DSML| | DSML markup token |
Roles
The encoding supports the following message roles: system, user, assistant, tool, latest_reminder, and developer.
Note on thedeveloperrole: Thedeveloperrole is used exclusively in the internal search agent pipeline. It is not needed for general-purpose chat or tool-calling tasks, and the official API does not accept messages with this role.
Basic Chat
A simple multi-turn conversation is encoded as:
{system_prompt}
{user_message}{response}
{user_message_2}{response_2}- The BOS token is prepended at the very beginning of the conversation.
- In chat mode (
thinking_mode="chat"),INLINECODE19zAECB6250to immediately close the thinking block, so the model generates content directly.
Interleaved Thinking Mode
In thinking mode (thinking_mode="thinking"), the model produces explicit reasoning inside ... blocks before responding.
{system_prompt}
{message}{reasoning}{response}The drop_thinking parameter (default True) controls whether reasoning from earlier turns is preserved:
- Without tools:
drop_thinkingtakes effect. Reasoning content from assistant turns before the last user message is stripped. Only the final assistant turn retains its...block. - With tools (on system or developer message):
drop_thinkingis automatically disabled. All turns retain their reasoning, because tool-calling conversations require full context for the model to track multi-step reasoning across tool calls.
Tool Calling (DSML Format)
Tools are defined on the system or developer message via the tools field (OpenAI-compatible format). When tools are present, the following schema block is injected into the system/user prompt:
INLINECODE32zD19D4085. For all other types (numbers, booleans, arrays, objects), pass the value in JSON format and set string="false".
If thinking_mode is enabled (triggered by ), you MUST output your complete reasoning inside ... BEFORE any tool calls or final response.
Otherwise, output directly after with tool calls or final response.
### Available Tool Schemas
{tool_definitions_json}
You MUST strictly follow the above defined tool name and parameter schemas to invoke tool calls.
An actual tool call in the assistant turn looks like:
INLINECODE35zD3963982string="true": the parameter value is a raw string.
string="false": the parameter value is JSON (number, boolean, array, object).
Tool execution results are wrapped in INLINECODE38z4EDC64FEtool_calls in the preceding assistant message.
Reasoning Effort
In thinking mode, the reasoning_effort parameter selects one of three levels, which control how much deliberation the model spends before answering. The level is realized purely as a text prefix prepended at the very beginning of the prompt (before the system message); the rest of the encoding is identical across levels.
reasoning_effort | Prompt prefix |
|---|---|
"low" (default) | none |
"high" | Reasoning Effort: Absolute maximum ... |
"max" | Reasoning Effort: Beyond maximum ... |
reasoning_effort has no effect in chat mode (thinking_mode="chat"), where the model does not produce a reasoning block at all.
The full prefix text for "high":
Reasoning Effort: Absolute maximum with no shortcuts permitted.
You MUST be very thorough in your thinking and comprehensively decompose the problem to resolve the root cause, rigorously stress-testing your logic against all potential paths, edge cases, and adversarial scenarios.
Explicitly write out your entire deliberation process, documenting every intermediate step, considered alternative, and rejected hypothesis to ensure absolutely no assumption is left unchecked.And for "max":
Reasoning Effort: Beyond maximum — exhaustive, relentless, and uncompromising.
You MUST reason with the utmost depth and rigor, leaving absolutely nothing to chance: exhaustively decompose the problem into its most fundamental components, trace every causal chain to its root, and resolve the underlying cause rather than any surface symptom.
Do not stop reasoning until you have independently verified the solution from multiple angles and are certain that no assumption remains unchecked and no error remains undiscovered.Quick Instruction Special Tokens
Quick instruction tokens are used for auxiliary classification and generation tasks. They are appended to messages via the "task" field to trigger specialized model behavior for a single-token or short-form output.
| Special Token | Description | Format |
|---|---|---|
INLINECODE52zB53B43BA...{prompt} | ||
INLINECODE54z1D53C305...{response} | ||
INLINECODE56zD2DA0746...{prompt} | ||
INLINECODE58z96E9D5EF...{prompt} | ||
INLINECODE60zD8A6C4A5...{prompt} | ||
INLINECODE62z6E5C219E | Determines whether each URL in the user prompt should be fetched and read. | ...{prompt}{url} |
Usage in message format:
actionon a user message: theINLINECODE66zF42F19D6query,authority,domain,read_url) on a user message: the task token is appended directly after the user content.titleon an assistant message: the `` token is appended after the assistant's EOS. The next assistant message provides the generated title.