GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
RedHatAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
RedHatAI
Model Tree
zai-org/GLM-5.3
Input Modalities
Output Modalities
Supported Functionality
This model is a quantized version of zai-org/GLM-5.3. It was evaluated on several tasks to assess its quality in comparison to the FP8 checkpoint.
This model was obtained by quantizing the weights and activations of zai-org/GLM-5.3 to FP4 (MXFP4, OCP Microscaling) data type, ready for inference with vLLM. Weights use FP4 (e2m1) with group size 32 and power-of-2 (E8M0) scales, while activations use FP4 with dynamic per-group scaling.
This optimization reduces the number of bits per parameter from 8 (FP8) to 4, reducing the disk size and GPU memory requirements by approximately 50%.
Only the weights and activations of the linear operators within transformer blocks are quantized using LLM Compressor.
text
vllm serve RedHatAI/GLM-5.3-MXFP4 \ --kv-cache-dtype fp8_e4m3 \ --tensor-parallel-size 4 \ --reasoning-parser glm45 \ --speculative-config '{"method":"mtp","num_speculative_tokens":5}'This model was created by applying LLM Compressor with the MXFP4 scheme in model-free post-training quantization (no calibration data required), exported in compressed-tensors format.
python
from compressed_tensors.entrypoints.convert import FP8BlockDequantizer from llmcompressor import model_free_ptq MODEL_ID = "zai-org/GLM-5.3"SAVE_DIR = MODEL_ID.rstrip("/").split("/")[-1] + "-MXFP4" ignore = [ "re:.*mlp.gate$", "re:.*lm_head", "re:.*embed_tokens$", "re:.*eh_proj$", "re:.*self_attn.indexer.weights_proj$",] model_free_ptq( model_stub=MODEL_ID, save_directory=SAVE_DIR, scheme="MXFP4", ignore=ignore + ["re:.*self_attn.indexer.wk$"], converter=FP8BlockDequantizer(ignore=ignore), max_workers=2, device="cuda:0",)This model was evaluated on GSM8K Platinum, IFEval, MATH-500, AIME 2025, and GPQA Diamond using lm-evaluation-harness and lighteval, all served with vLLM (OpenAI-compatible API). Each benchmark was run with 3 seeds (1234, 2345, 3456) and the results averaged; recovery is computed against the FP8 checkpoint on 4 B200s.
vllm serve RedHatAI/GLM-5.3-MXFP4 \ --kv-cache-dtype fp8_e4m3 \ --tensor-parallel-size 4 \ --reasoning-parser glm45 \ --speculative-config '{"method":"mtp","num_speculative_tokens":5}'from compressed_tensors.entrypoints.convert import FP8BlockDequantizer from llmcompressor import model_free_ptq MODEL_ID = "zai-org/GLM-5.3"SAVE_DIR = MODEL_ID.rstrip("/").split("/")[-1] + "-MXFP4" ignore = [ "re:.*mlp.gate$", "re:.*lm_head", "re:.*embed_tokens$", "re:.*eh_proj$", "re:.*self_attn.indexer.weights_proj$",] model_free_ptq( model_stub=MODEL_ID, save_directory=SAVE_DIR, scheme="MXFP4", ignore=ignore + ["re:.*self_attn.indexer.wk$"], converter=FP8BlockDequantizer(ignore=ignore), max_workers=2, device="cuda:0",)