DeepSeek-V4-Flash-0731

Engine
vLLM
Listed
3 Nov 2026

This document describes the prompt encoding format used by DeepSeek-V4 series models. The encoding handles multi-turn conversations, tool calling, extended thinking (reasoning), and quick instruction tasks.

Serves
Chat
Parameters
83.4B
Context
1,048,576 tokens
Weights
167 GB
License
MIT

Capabilities

Read from the model's capability tags. Requests are admitted per tag.

Chat chat
  • POST /v1/chat/completions
  • POST /v1/responses

Endpoint task mode generate

Inputs
Text
Reasoning effort reasoning_effort
does not reason
Structured output structured_output
supported
Proven by probe: a JSON schema request returned a payload that parses.
Forced tool choice forced_tool_choice
supported
A forced tool_choice returns a call to that tool.

Launch settings

Every engine serving the model starts with these.

Set by the owner. auto: the serving node derives it. default: the platform's value, left in place.

Engine
vLLM
Context window --max-model-len
1,048,576 tokens
Quantization --quantization
none default
Weights load as stored, fp8.
KV cache dtype --kv-cache-dtype
auto
No flag: the checkpoint's KV quantization, else the model dtype.
Scheduling policy --scheduling-policy
priority default
Interactive requests ahead of batch work; same-class traffic in arrival order.
Tool-call parser --tool-call-parser
auto
No parser for this architecture.
Reasoning parser --reasoning-parser
auto
No parser for this architecture.
Sampling defaults generation_config.json
temperature 1, top_p 1
Used when a request leaves them out.
Trust remote code --trust-remote-code
on default
Python shipped in the bundle may run at load (the default).

Engine tuning

Every setting auto: the serving node derives each one.

Set by the owner. auto: the serving node derives it.

Mixture of experts

Expert parallel --enable-expert-parallel
auto
Expert load balancing --enable-eplb
auto
All-to-all backend --all2all-backend
auto

Memory and batching

GPU memory utilization --gpu-memory-utilization
auto
Max sequences --max-num-seqs
auto
Max batched tokens --max-num-batched-tokens
auto
Long-prefill threshold --long-prefill-token-threshold
auto

Execution

Enforce eager --enforce-eager
auto
Prefix caching --enable-prefix-caching
auto
Async scheduling --no-async-scheduling
auto
Custom all-reduce --disable-custom-all-reduce
auto

Speculative decoding

Draft tokens checked by the model in one pass.

Speculative decoding
not configured
An endpoint's own setting, or the serving node's default for the model family, applies.

Architecture

Read from the model's files at upload.

Parameters
83,439,268,220
Layers num_hidden_layers
43
Attention heads num_attention_heads
64
Head dim head_dim
512
Experts
256
Vocabulary vocab_size
129,280
Native dtype torch_dtype
fp8
Weights on disk
167 GB, safetensors

Call it

Create an endpoint for the model, then send it a request in the OpenAI shape.

Shell
# Create an endpoint for this model, then provision it. # --tier-id is the service tier to run on; pick one at /endpoints/new. erectl endpoints create \ --name "My endpoint" \ --slug my-endpoint \ --model-id 9E650D01-355E-4E8F-8CBB-35D09D69D2F7 \ --tier-id <tier-id> # provision takes the endpoint UUID returned by create erectl endpoints provision <endpoint-id> # Call it. Requests are routed by /<project-id>/<endpoint-slug>. curl https://api.erebine.ai/proj_ABC123/my-endpoint/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $EREBINE_API_KEY" \ -d '{ "model": "DeepSeek-V4-Flash-0731", "messages": [{"role": "user", "content": "Hello, world!"}], "stream": true }'

Model card

Images are left out; its HTML tables are set as tables.

DeepSeek-V4 Encoding

A self-contained reference implementation is provided in encoding_dsv4.py.

Quick Start

from encoding_dsv4 import encode_messages, parse_message_from_completion_text

# Encode a conversation
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is 2+2?"},
]
prompt = encode_messages(messages, thinking_mode="thinking")
# => "You are a helpful assistant.What is 2+2?"

# Parse model output back to structured message
completion = "Simple arithmetic.2 + 2 = 4."
parsed = parse_message_from_completion_text(completion, thinking_mode="thinking")
# => {"role": "assistant", "reasoning_content": "Simple arithmetic.", "content": "2 + 2 = 4.", "tool_calls": []}
Note: The parse_message_from_completion_text function is designed to handle well-formatted model output only. It does not attempt to correct or recover from malformed output that the model might occasionally generate. For production use, additional error handling is recommended.

Message Format

Special Tokens
TokenPurpose
INLINECODE2zE74A5889End of assistant turn (EOS)
INLINECODE4zACED29E5Assistant turn prefix
INLINECODE6zEF441620 / INLINECODE8zAD297779|DSML|DSML markup token
Roles

The encoding supports the following message roles: system, user, assistant, tool, latest_reminder, and developer.

Note on the developer role: The developer role is used exclusively in the internal search agent pipeline. It is not needed for general-purpose chat or tool-calling tasks, and the official API does not accept messages with this role.
Basic Chat

A simple multi-turn conversation is encoded as:

{system_prompt}
{user_message}{response}
{user_message_2}{response_2}
  • The BOS token is prepended at the very beginning of the conversation.
  • In chat mode (thinking_mode="chat"), INLINECODE19zAECB6250 to immediately close the thinking block, so the model generates content directly.
Interleaved Thinking Mode

In thinking mode (thinking_mode="thinking"), the model produces explicit reasoning inside ... blocks before responding.

{system_prompt}
{message}{reasoning}{response}

The drop_thinking parameter (default True) controls whether reasoning from earlier turns is preserved:

  • Without tools: drop_thinking takes effect. Reasoning content from assistant turns before the last user message is stripped. Only the final assistant turn retains its ... block.
  • With tools (on system or developer message): drop_thinking is automatically disabled. All turns retain their reasoning, because tool-calling conversations require full context for the model to track multi-step reasoning across tool calls.
Tool Calling (DSML Format)

Tools are defined on the system or developer message via the tools field (OpenAI-compatible format). When tools are present, the following schema block is injected into the system/user prompt:

INLINECODE32zD19D4085. For all other types (numbers, booleans, arrays, objects), pass the value in JSON format and set string="false". If thinking_mode is enabled (triggered by ), you MUST output your complete reasoning inside ... BEFORE any tool calls or final response. Otherwise, output directly after with tool calls or final response. ### Available Tool Schemas {tool_definitions_json} You MUST strictly follow the above defined tool name and parameter schemas to invoke tool calls.


An actual tool call in the assistant turn looks like:
xml string_value 5 INLINECODE35zD3963982string="true": the parameter value is a raw string.
  • string="false": the parameter value is JSON (number, boolean, array, object).

Tool execution results are wrapped in INLINECODE38z4EDC64FEtool_calls in the preceding assistant message.

Reasoning Effort

In thinking mode, the reasoning_effort parameter selects one of three levels, which control how much deliberation the model spends before answering. The level is realized purely as a text prefix prepended at the very beginning of the prompt (before the system message); the rest of the encoding is identical across levels.

reasoning_effortPrompt prefix
"low" (default)none
"high"Reasoning Effort: Absolute maximum ...
"max"Reasoning Effort: Beyond maximum ...

reasoning_effort has no effect in chat mode (thinking_mode="chat"), where the model does not produce a reasoning block at all.

The full prefix text for "high":

Reasoning Effort: Absolute maximum with no shortcuts permitted.
You MUST be very thorough in your thinking and comprehensively decompose the problem to resolve the root cause, rigorously stress-testing your logic against all potential paths, edge cases, and adversarial scenarios.
Explicitly write out your entire deliberation process, documenting every intermediate step, considered alternative, and rejected hypothesis to ensure absolutely no assumption is left unchecked.

And for "max":

Reasoning Effort: Beyond maximum — exhaustive, relentless, and uncompromising.
You MUST reason with the utmost depth and rigor, leaving absolutely nothing to chance: exhaustively decompose the problem into its most fundamental components, trace every causal chain to its root, and resolve the underlying cause rather than any surface symptom.
Do not stop reasoning until you have independently verified the solution from multiple angles and are certain that no assumption remains unchecked and no error remains undiscovered.
Quick Instruction Special Tokens

Quick instruction tokens are used for auxiliary classification and generation tasks. They are appended to messages via the "task" field to trigger specialized model behavior for a single-token or short-form output.

Special TokenDescriptionFormat
INLINECODE52zB53B43BA...{prompt}
INLINECODE54z1D53C305...{response}
INLINECODE56zD2DA0746...{prompt}
INLINECODE58z96E9D5EF...{prompt}
INLINECODE60zD8A6C4A5...{prompt}
INLINECODE62z6E5C219EDetermines whether each URL in the user prompt should be fetched and read....{prompt}{url}

Usage in message format:

  • action on a user message: the INLINECODE66zF42F19D6query, authority, domain, read_url) on a user message: the task token is appended directly after the user content.
  • title on an assistant message: the `` token is appended after the assistant's EOS. The next assistant message provides the generated title.