Architecture
Glm5NextForConditionalGeneration (glm5_next), multimodal (image-text-to-text)
- 45 decoder layers + 1 MTP layer, hidden size 4096, 288 routed experts
- 1,048,576 context
- compressed-tensors
nvfp4-pack-quantized, group size 16
Serving (vLLM)
vllm serve OpenYourMind/GLM-5.3-Flash-NVFP4-sel3-MTP \ --trust-remote-code \ --tensor-parallel-size 4 \ --tool-call-parser glm47 \ --enable-auto-tool-choice \ --reasoning-parser glm45 \ --enable-prefix-caching \ --speculative-config '{"method":"mtp","num_speculative_tokens":1}'
Drop the --speculative-config line to run without speculative decoding.
Requires an NVFP4-capable GPU (Blackwell / sm100+) for the native FP4 path.
Limitations
- Alignment behaviour differs from the base model. Use accordingly.
- Quantization is NVFP4; expect small deviations from the BF16 base.
Attribution
Derived from zai-org/GLM-5.3-Flash (MIT).