GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
intel-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
intel-ai
Model Tree
zai-org/GLM-5.3-Flash-BF16
Input Modalities
Output Modalities
Supported Functionality
This model is an MXFP4 (W4A4, group_size 32) model of zai-org/GLM-5.3-Flash generated by intel/auto-round using RTN. Please follow the license of the original model.
bash
## Using the latest vLLM version should work fine.uv pip install "vllm @ git+https://github.com/vllm-project/vllm@46d2b23ac5047a813ebb082122166e4ae09b5f39"bash
vllm serve intel-ai/GLM-5.3-Flash-MXFP4-AutoRound \ --kv-cache-dtype fp8 \ --dtype bfloat16| Benchmark | BF16 | MXFP4 | Recovery |
|---|---|---|---|
| PIQA | 82.92 | 82.75 | 99.79% |
| HellaSwag | 69.03 | 68.25 | 98.87% |
| MMLU | 86.66 | 85.88 | 99.10% |
| GSM8K | 97.35 | 97.50 | 100.15% |
| Average | 83.99 | 83.59 | 99.53% |
bash
auto-round \ --model_name zai-org/GLM-5.3-Flash-BF16 \ --model_free \ --scheme MXFP4 \ --ignore_layers "lm_head,embed_tokens,visual,indexer,weights_proj,self_attn,mlp.shared_experts,layers.45,eh_proj,enorm,hnorm,shared_head,norm,A_log,dt_bias,conv1d" \ --layer_config '{"model.language_model.layers.0.mlp.gate_proj":{"bits":4,"data_type":"mx_fp"},"model.language_model.layers.0.mlp.up_proj":{"bits":4,"data_type":"mx_fp"},"model.language_model.layers.0.mlp.down_proj":{"bits":4,"data_type":"mx_fp"}}' \ --format llm_compressor \ --output_dir GLM-5.3-Flash-BF16-MXFP4-Mixed-CT-RTN-AutoRoundThe model inherits the capabilities, biases, and safety behavior of the base checkpoint. Quantization does not add alignment, and reduced numerical precision can change outputs on edge cases. No additional training data was used.
--kv-cache-dtype fp8, matching the reference checkpoint's FP8 KV-cache setting.language_model.model.*, visual.*, language_model.lm_head, plus RoutedExperts for the fused MoE); serving through a runtime that expects the raw Hugging Face module names requires retargeting the config.This model is a derivative of GLM-5.3-Flash and is provided under the same MIT license. It is not an official Z.ai or NVIDIA release.
bibtex
@software{autoround, title = {AutoRound: Advanced Weight-Only Quantization for LLMs}, author = {Intel}, url = {https://github.com/intel/auto-round}}Signed-off-by: Kaokao, Lv kaokao.lv@intel.com
## Using the latest vLLM version should work fine.uv pip install "vllm @ git+https://github.com/vllm-project/vllm@46d2b23ac5047a813ebb082122166e4ae09b5f39"vllm serve intel-ai/GLM-5.3-Flash-MXFP4-AutoRound \ --kv-cache-dtype fp8 \ --dtype bfloat16auto-round \ --model_name zai-org/GLM-5.3-Flash-BF16 \ --model_free \ --scheme MXFP4 \ --ignore_layers "lm_head,embed_tokens,visual,indexer,weights_proj,self_attn,mlp.shared_experts,layers.45,eh_proj,enorm,hnorm,shared_head,norm,A_log,dt_bias,conv1d" \ --layer_config '{"model.language_model.layers.0.mlp.gate_proj":{"bits":4,"data_type":"mx_fp"},"model.language_model.layers.0.mlp.up_proj":{"bits":4,"data_type":"mx_fp"},"model.language_model.layers.0.mlp.down_proj":{"bits":4,"data_type":"mx_fp"}}' \ --format llm_compressor \ --output_dir GLM-5.3-Flash-BF16-MXFP4-Mixed-CT-RTN-AutoRound@software{autoround, title = {AutoRound: Advanced Weight-Only Quantization for LLMs}, author = {Intel}, url = {https://github.com/intel/auto-round}}