GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
JANGQ-AI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
JANGQ-AI
Model Tree
zai-org/GLM-5.3-Flash
Input Modalities
Output Modalities
Supported Functionality
| Module | Source dtype | Bundle dtype |
|---|---|---|
| Routed experts (288 × 3 × MoE layers 3–44) | BF16 | INT4 sym g32 (weight_packed/weight_scale/weight_shape) |
Vision tower (model.visual.*) | BF16 | BF16 |
Attention (full + linear A_log/dt_bias, indexer) | BF16 | BF16 |
| Shared expert, dense MLP (layers 0–2) | BF16 | BF16 |
Router gates, norms, embed, lm_head, MTP (layer 45) | BF16 / F32 | BF16 / F32 |
Dequant per group of 32: w = int4 * weight_scale (scale = amax/7.5, range [-8,7]).
bash
vllm serve JANGQ-AI/GLM-5.3-Flash-W4A16 --tensor-parallel-size 4 --trust-remote-code
Runtime note:
glm5_nextis a very new hybrid multimodal architecture. Confirm your serving stack (vLLM / SGLang build) hasGlm5Nextsupport before deploying; the weights here are standard compressed-tensors W4A16 and load via the Marlin path once the architecture is recognized.
Streaming shard-by-shard RTN quantizer (one tensor in memory at a time — no GPU,
no large host RAM), architecture-agnostic: the routed-expert tensor set is taken
from the model's own index, packed via compressed_tensors.pack_to_int32 for a
bit-exact on-disk layout. Data-free (no calibration). Every layer verified present
(0–45), MTP + vision preserved, expert count checked against the source index.
Quantized by JANGQ-AI.