GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Jon-Nielsen
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Jon-Nielsen
Model Tree
zai-org/GLM-4.7
Input Modalities
Output Modalities
Supported Functionality
This is a 4-bit AutoRound GPTQ quantization of 0xSero/GLM-4.7-REAP-30, a 30% REAP-pruned version of the original GLM-4.7 MoE model.
bits=4, group_size=128, sym=True) in auto_gptq format.(tested on 2x NVIDIA RTX PRO 6000 Blackwell with the following settings)
bash
vllm serve Jon-Nielsen/GLM-4.7-REAP-30-W4A16 \ --tensor-parallel-size 2 \ --kv-cache-dtype fp8 \ --gpu-memory-utilization 0.96 \ --max-model-len 196608 \ --max-num-seqs 8 \ --max-num-batched-tokens 16384 \ --trust-remote-code \ --enable-prefix-caching \ --enable-chunked-prefill \ --enable-expert-parallel \ --tool-call-parser=glm47 \ --reasoning-parser=glm45 \ --enable-auto-tool-choice \ --disable-custom-all-reduce