GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
kevincuicsx
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
kevincuicsx
Model Tree
zai-org/GLM-5.2
Input Modalities
Output Modalities
Supported Functionality
This checkpoint is a streaming GPTQ quantization of zai-org/GLM-5.2. Only routed MoE expert linear weights and their activations are NVFP4; attention, shared experts, routers, dense layers, embeddings, and the language-model head remain BF16. NVFP4 uses E2M1 weights and activations, FP8 E4M3 block scales with group size 16, and FP32 tensor scales.
Quantization used 64 coding-oriented calibration sequences of 1024
tokens, damp=0.01, GPTQ block size=128, and expert Hessian shrinkage
tau=1024.0. The source revision is b4734de4facf877f85769a911abafc5283eab3d9 and the calibration token SHA256 is
0bd4f9e13a6714335a83225915efe43855fb6351ba1f17c27f99b9e2aeda6b1f.
The checkpoint follows NVIDIA ModelOpt's serialized NVFP4 layout and is intended for B200-class GPUs:
bash
vllm serve kevincuicsx/GLM-5.2-GPTQ-NVFP4 --tensor-parallel-size 8 --quantization modelopt_fp4
Quantization implementation: Together's tquark GLM-5.2 streaming GPTQ driver. KV cache quantization is disabled.