GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
eigen-ai-labs
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
eigen-ai-labs
Model Tree
zai-org/GLM-5.2
Input Modalities
Output Modalities
Supported Functionality
GLM-5.2-NVFP4 is an NVFP4-quantized version of zai-org/GLM-5.2, a 744B-parameter Mixture-of-Experts language model with 40B active parameters, 256 experts per MoE layer (8 activated per token), and DeepSeek Sparse Attention (DSA).
Quantized directly from the full BF16 checkpoint (zai-org/GLM-5.2, not the FP8 release, to NVFP4 (4-bit with blockwise FP8 scales per 16 elements) using NVIDIA Model Optimizer.
Only the non-shared MoE expert MLP projections are quantized to NVFP4. Attention weights are left in BF16, in addition to the dense MLPs (layers 0-3) and the shared experts. Since the MoE expert weights constitute the vast majority of model parameters in an MoE architecture, this still yields significant memory savings.
Calibration uses natural top-k routing rather than forcing all experts to activate, so each expert's quantization scales reflect the token distributions it actually sees during inference. To compensate, calibration was run on a much larger number of samples than typical to ensure broad expert coverage through natural routing alone.
Three calibration passes were run:
Hardware: 8x RTX PRO 6000 Blackwell 96GB (b12x MoE runner recommended)