GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
JANGQ-AI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
JANGQ-AI
Model Tree
zai-org/GLM-5.3
Input Modalities
Output Modalities
Supported Functionality
| Module | Source dtype | Bundle dtype |
|---|---|---|
Routed experts (experts.*.{gate,up,down}_proj, incl. MTP) | BF16 | INT4 sym g32 (weight_packed i32 / weight_scale bf16 / weight_shape i64) |
| Attention (MLA q/kv proj, o_proj, DSA indexer) | BF16 | BF16 |
| Dense MLP (first 3 layers) + shared expert | BF16 | BF16 |
Router gates, e_score_correction_bias, RMSNorms | BF16 / F32 | BF16 / F32 |
embed_tokens, lm_head, MTP glue (eh_proj, norms) | BF16 | BF16 |
Dequant per group of 32: w = int4 * weight_scale (symmetric, scale = amax/7.5,
range [-8, 7]).
bash
vllm serve JANGQ-AI/GLM-5.3-W4A16 \ --tensor-parallel-size 8 \ --trust-remote-codevLLM auto-selects the Marlin MoE kernel for compressed-tensors pack-quantized INT4 at group-32 on Hopper.
Converted from the BF16 release with a streaming shard-by-shard quantizer (one
tensor in memory at a time — no GPU, no large host RAM). Round-to-nearest INT4
is packed via compressed_tensors.pack_to_int32 so the layout is bit-exact with
what vLLM expects; the format was ground-truthed against a known-working
DeepSeek-family W4A16 model. This is a data-free RTN quant (no calibration).
For maximum quality a calibrated GPTQ/AWQ pass on the routed experts can be run
later on-device.
Quantized by JANGQ-AI.