GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
shankinsonhf
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
shankinsonhf
Model Tree
zai-org/GLM-5.3-Flash-BF16
Input Modalities
Output Modalities
Supported Functionality
43fd41a58d52c4e6e5dec1d1ff5989ecc737ae1agate_proj, up_proj, and down_proj: 36,288 weights totallm_head, and norms remain BF16The activation-scale policy uses a 5% safety margin with:
w2 at layers 3–8w2 at layers 24–44The conversion and active-scale hashes are recorded in
conversion_manifest.json. The installed activation-scale shard SHA-256 is
caacc401233a4d6bcae6fc4f830c1754b12f28d0f6c5b513ceb1e586b40f5eb1.
The final clean-cold four-node run used vLLM
0.1.dev20051+g487ecf187, FlashInfer 0.6.17, CUDA sm_121a, native
FLASHINFER_CUTLASS NVFP4 MoE, and FLASHINFER_MLA_SPARSE_SM120.
Constrained 400-item MMLU/ScienceQA regression evaluation:
| Checkpoint | Overall | Text | Image-text |
|---|---|---|---|
| Official FP8 reference | 89.25% | 84.5% | 94.0% |
| W4A16 activation reference | 88.5% | 83.0% | 94.0% |
| This W4A4 checkpoint | 89.0% | 84.0% | 94.0% |
Unprofiled W4A4 phase throughput on four GB10 nodes:
| Concurrent width | Prefill prompt tok/s | Generation output tok/s |
|---|---|---|
| 1 | 1,279.01 | 24.86 |
| 2 | 1,332.22 | 40.63 |
| 4 | 1,334.71 | 65.73 |
| 8 | 1,383.55 | 99.77 |
| 16 | 1,382.60 | 173.77 |
The long-prompt prefill case generated one token. The decode-dominant case used a short prompt and 256 forced output tokens. Exact deterministic output, reasoning, tools, image, four-image, 32-frame video, streaming, 7,820-token and 31,020-token prompts passed. A 901.5-second mixed-width soak completed 2,495 requests without failures or earlyoom intervention.
bash
hf download shankinsonhf/GLM-5.3-Flash-NVFP4-W4A4-Experts \--local-dir /path/to/GLM-5.3-Flash-NVFP4-W4A4-Experts
The checkpoint is approximately 191 GiB (205 GB decimal). The repository uses 120 primary safetensors shards plus a separate activation-scale shard.
Use this checkpoint only with a runtime that recognizes ModelOpt
MIXED_PRECISION metadata and native NVFP4 expert layers. The validated GB10
deployment requires the
companion vLLM compatibility patches and launch settings;
vanilla vLLM behavior has not been claimed or validated. The checkpoint
retains the upstream multimodal processor, tokenizer, chat template, and MIT
license.
This conversion is not affiliated with Z.AI, NVIDIA, vLLM, or FlashInfer.