GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AxionML
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
AxionML
Model Tree
deepseek-ai/DeepSeek-V4.1-Flash
Input Modalities
Output Modalities
Supported Functionality
| Architecture | Causal Encoder-Decoder MoE with Compressed Sparse Attention 2 (DeepseekV41ForCausalLM) |
| Backbone Parameters | 552B total |
| Activated Parameters | 8B (prefill) / 16B (decode) |
| Engram Memory | 196B |
| Experts | 384 routed, 6 active, 40 layers |
| Input | Text, image |
| Context Length | 1M tokens |
| Checkpoint Size | ~527 GB |
| Benchmark | MXFP4 (source) | NVFP4 |
|---|---|---|
| GPQA Diamond | 91.035 | 91.288 |
| AA-LCR | 78.563 | 78.438 |
| SciCode | 54.401 | 55.843 |
| IFBench | 76.667 | 77.267 |
| MMMU-Pro | 74.046 | 73.699 |
| Terminal-Bench 2.1 | 81.60 |
Scores reported by NVIDIA for this checkpoint (vLLM). Baseline: deepseek-ai/DeepSeek-V4.1-Flash.
temperature=1.0,top_p=0.95,reasoning_effort=100.
w1, w2, w3) converted from source MXFP4 to NVFP4 W4A4 (group size 16); attention, shared experts, vision, Engram tables and MTP/DSpark keep their source precision (incl. MXFP8)bash
python -m sglang.launch_server \ --model-path AxionML/DeepSeek-V4.1-Flash-NVFP4 \ --tp 4 \ --context-length 1048576 \ --reasoning-parser deepseek-v41 \ --tool-call-parser deepseekv41 \ --chunked-prefill-size 4096 \ --max-running-requests 16bash
vllm serve AxionML/DeepSeek-V4.1-Flash-NVFP4 \ --tensor-parallel-size 4 \ --tokenizer-mode deepseek_v41 \ --reasoning-parser deepseek_v41 \ --language-model-only \ --max-model-len 1048576 \ --max-num-seqs 32 \ --max-num-batched-tokens 8192 \ --enable-chunked-prefill \ --no-enable-prefix-cachingValidated upstream on 4x GB300 with lmsysorg/sglang:dev-cu13-dsv41 and vllm/vllm-openai:deepseekv41-flash-0909. Pass a request-level reasoning_effort (e.g. "max") to enable thinking in SGLang. DSpark tensors are preserved but speculative decoding was not validated.
The base model was trained on data that may contain toxic language and societal biases. The quantized model inherits these limitations. It may generate inaccurate, biased, or offensive content. Please refer to the original model card and the upstream quantized model card for full details.
| 82.16 |