GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AxionML
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
AxionML
Model Tree
Qwen/Qwen3.5-397B-A17B
Input Modalities
Output Modalities
Supported Functionality
| Architecture | Hybrid-attention MoE (Gated DeltaNet + Gated Attention) with vision encoder |
| Total Parameters | 397B |
| Activated Parameters | 17B |
| Context Length | 262K tokens |
| Checkpoint Size | ~244 GB |
| Benchmark | FP8 (baseline) | NVFP4 (V2) |
|---|---|---|
| MMMU Pro | 78.7 | 78.4 |
| GPQA Diamond | 87.2 | 87.7 |
| SciCode | 46.7 | 48.1 |
| AA-LCR | 68.8 | 67.8 |
| IFBench | 76.1 | 76.5 |
| τ²-Bench Telecom | 95.4 | 95.2 |
Scores reported by NVIDIA for this checkpoint. Baseline: Qwen/Qwen3.5-397B-A17B-FP8.
temperature=0.6,top_p=0.95, max 64,000 tokens (128,000 for τ²-Bench Telecom).
nvidia/Qwen3.5-397B-A17B-NVFP4, Feb 2026) quantized only the routed experts with max calibration and left attention and shared experts in BF16 (~251 GB). V2 adds MSE scale search and FP8 attention/shared experts — smaller and faster, and within noise of Qwen's FP8 checkpoint.bash
python3 -m sglang.launch_server \ --model-path AxionML/Qwen3.5-397B-A17B-NVFP4 \ --tensor-parallel-size 4 \ --quantization modelopt_mixed \ --disable-radix-cache \ --trust-remote-codeValidated upstream with lmsysorg/sglang:v0.5.12.post1-cu130 on B300. Tip: add --mamba-ssm-dtype bfloat16 to keep the Mamba SSM state in BF16 for faster decode on Blackwell.
The base model was trained on data that may contain toxic language and societal biases. The quantized model inherits these limitations. It may generate inaccurate, biased, or offensive content. Please refer to the original model card and the upstream quantized model card for full details.
python3 -m sglang.launch_server \ --model-path AxionML/Qwen3.5-397B-A17B-NVFP4 \ --tensor-parallel-size 4 \ --quantization modelopt_mixed \ --disable-radix-cache \ --trust-remote-code