GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
dealignai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
dealignai
Model Tree
deepseek-ai/DeepSeek-V4-Flash-0731
Input Modalities
Output Modalities
Supported Functionality
deepseek-ai/DeepSeek-V4-Flash-0731 (43-layer MoE, 256 routed + 1 shared expert, MLA,
sliding-window attention, ~1M context via YaRN). Native quant: FP8 (attention + shared experts)
| Metric | Base | This model |
|---|---|---|
| MMLU (logit, 285Q) | 84.2% | 86.7% (+2.5pp) |
| HumanEval pass@2 (70) | — | 94.3% |
| HarmBench compliance (240) | refuses | ≥96%, 0 hard refusals |
| Decode throughput (single-stream) | — | 45–49 tok/s |
| Prefix-cache hit (12-way concurrent) | — | 64% (3.7× faster on hit) |
| Multiturn correctness (24 concurrent) | — | 24/24 |
| GPU KV cache | — | 1.27M tokens (fp8) |
240 behaviors, answer-channel (no-refusal) compliance, greedy:
| Category | Comply | Rate |
|---|---|---|
| chemical biological | 42/42 | 100% |
| cybercrime intrusion | 52/52 | 100% |
| harassment bullying | 21/21 | 100% |
| harmful | 18/18 | 100% |
| illegal | 53/53 | 100% |
| misinformation disinformation | 54/54 |
Knowledge and reasoning are fully retained — MMLU is within noise (slightly higher). Compliance is measured on the answer channel.
Official DeepSeek-V4-Flash defaults, stamped as this repo's generation_config.json:
| Param | Value |
|---|---|
temperature | 1.0 |
top_p | 0.95 |
json
{"do_sample": true, "temperature": 1.0, "top_p": 0.95}
These are applied automatically when a request omits sampling params. Override per-request as needed.
Requires the DeepSeek-V4 vLLM path for sm_121 (e.g. the jasl/vllm GB10 build). Head + worker
over a RoCE point-to-point link, TP=2 + expert-parallel, fp8 KV cache, DSpark spec-decode, 384K ctx.
See the included serve/ scripts. Key flags:
markdown
vllm serve <this-model> --tensor-parallel-size 2 --enable-expert-parallel \--distributed-executor-backend mp --nnodes 2 --node-rank {0,1} --master-addr <head> \--kv-cache-dtype fp8 --block-size 256 --enable-prefix-caching \--max-model-len 393216 --gpu-memory-utilization 0.80 \--speculative-config '{"method":"dspark","num_speculative_tokens":5}' \--reasoning-parser deepseek_v4 --tool-call-parser deepseek_v4 --enable-auto-tool-choice
Controllable thinking (low / high / max) via the reasoning system preface; the reasoning trace is
returned as reasoning_content, the answer as content. Note: thinking mode reasons at length —
allow ≥2500 output tokens or the answer may be truncated.
DeepSeek-V4 tool-call parser; pass OpenAI-style tools to the chat endpoint.
MIT (inherits the upstream DeepSeek-V4-Flash license terms). Review before use.
| 100% |