GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AxionML
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
AxionML
Model Tree
deepseek-ai/DeepSeek-V4-Flash
Input Modalities
Output Modalities
Supported Functionality
| Architecture | MoE with hybrid attention (Compressed Sparse + Heavily Compressed Attention), Manifold-Constrained Hyper-Connections |
| Total Parameters | 284B |
| Activated Parameters | 13B |
| Reasoning Modes | Non-think / Think High / Think Max |
| Context Length | 1M tokens |
| Checkpoint Size | ~168 GB |
| Benchmark | Baseline | NVFP4 |
|---|---|---|
| GPQA Diamond | 89.4 | 89.1 |
| AA-LCR | 65.8 | 65.5 |
| τ²-Bench Telecom | 94.3 | 94.2 |
| SciCode | 48.1 | 48.1 |
| IFBench | 78.8 | 79.5 |
Scores reported by NVIDIA for this checkpoint.
temperature=1.0,top_p=1.0, max 384,000 tokens.
bash
python3 -m sglang.launch_server \ --model-path AxionML/DeepSeek-V4-Flash-NVFP4 \ --tensor-parallel-size 8 \ --trust-remote-codebash
vllm serve AxionML/DeepSeek-V4-Flash-NVFP4 \ --tensor-parallel-size 4 \ --trust-remote-code \ --kv-cache-dtype fp8SGLang auto-detects NVFP4 from hf_quant_config.json (sgl-project/sglang#25820). vLLM validated upstream on GB300 (TP4). Superseded by AxionML/DeepSeek-V4-Flash-0731-NVFP4.
The base model was trained on data that may contain toxic language and societal biases. The quantized model inherits these limitations. It may generate inaccurate, biased, or offensive content. Please refer to the original model card and the upstream quantized model card for full details.
python3 -m sglang.launch_server \ --model-path AxionML/DeepSeek-V4-Flash-NVFP4 \ --tensor-parallel-size 8 \ --trust-remote-codevllm serve AxionML/DeepSeek-V4-Flash-NVFP4 \ --tensor-parallel-size 4 \ --trust-remote-code \ --kv-cache-dtype fp8