GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
incoai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
incoai
Model Tree
zai-org/GLM-5.3
Input Modalities
Output Modalities
Supported Functionality
This checkpoint was produced with our in-house post-training quantization
toolchain and exported in the
NVIDIA Model Optimizer
format. Only the weights and activations of the linear
operators within the routed MoE experts are quantized to NVFP4, with static
per-tensor activation scales. The KV cache is quantized to FP8 with static
unit scales (kv_cache_quant_algo: FP8).
The commands below enable speculative decoding with the
DFlash 2 draft model
(lossless, 7 draft tokens per verification step); remove the
--speculative-* flags to serve without it.
Serve with SGLang (main):
bash
python3 -m sglang.launch_server \ --model-path incoai/GLM-5.3-NVFP4 \ --tp 8 \ --quantization modelopt_fp4 \ --reasoning-parser glm45 \ --tool-call-parser glm47 \ --chunked-prefill-size 8192 \ --speculative-algorithm DFLASH \ --speculative-draft-model-path incoai/GLM-5.3-DFlash2 \ --speculative-draft-attention-backend trtllm_mhaOr with vLLM (v0.28.0 or later):
bash
vllm serve incoai/GLM-5.3-NVFP4 \ --tensor-parallel-size 8 \ --reasoning-parser glm45 \ --tool-call-parser glm47 \ --enable-auto-tool-choice \ --speculative-config '{"method":"dflash","model":"incoai/GLM-5.3-DFlash2","num_speculative_tokens":7}'We compare Z.ai's original FP8 release with this NVFP4 checkpoint. Higher is better.
| Precision | GPQA Diamond | AIME 2025 | MATH-500 | HLE | AA-LCR |
|---|---|---|---|---|---|
| FP8 | 91.1 | 94.3 | 95.6 | 35.9 | 73.6 |
| NVFP4 | 91.2 | 95.1 | 95.2 | 35.2 | 73.0 |
This model is a quantized version of GLM-5.3 and is distributed under Z.ai's GLM-5.3 License, which it inherits from the base model.
python3 -m sglang.launch_server \ --model-path incoai/GLM-5.3-NVFP4 \ --tp 8 \ --quantization modelopt_fp4 \ --reasoning-parser glm45 \ --tool-call-parser glm47 \ --chunked-prefill-size 8192 \ --speculative-algorithm DFLASH \ --speculative-draft-model-path incoai/GLM-5.3-DFlash2 \ --speculative-draft-attention-backend trtllm_mhavllm serve incoai/GLM-5.3-NVFP4 \ --tensor-parallel-size 8 \ --reasoning-parser glm45 \ --tool-call-parser glm47 \ --enable-auto-tool-choice \ --speculative-config '{"method":"dflash","model":"incoai/GLM-5.3-DFlash2","num_speculative_tokens":7}'