GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Mr-ezdno
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Mr-ezdno
Model Tree
Blackfrost-AI/RED-SNOW-5.3-FLASH-BF16
Input Modalities
Output Modalities
Supported Functionality
| Part | Source |
|---|---|
| EXL3 K2 experts, codebook mcg, 43 MoE layers + MTP layer 45 | own exllamav3 1.6.2 conversion of the RED-SNOW BF16 weights (-b 2.0 -hq -hb 5 -mb 2 -cb mcg --out_scales always, calibration 250×2048) |
| Trunk (attention / shared / dense / lm_head / embed / vision), affine 5b/6b g128 | rebuilt from the RED-SNOW BF16 checkpoint (HTTP Range) |
| Pack skeleton, tokenizer | the beamster/GLM-5.3-Flash-Sushi-2bpw template (MIT, Z.AI) |
DFlash2 drafter (GLM-5.3-Flash-DFlash2/) | Inco.ai's incoai/GLM-5.3-Flash-DFlash2, unmodified, via the same template (CC BY-NC-ND 4.0) |
chat_template.jinja, generation_config.json | RED-SNOW (the model's built-in persona lives in the template; client system prompts merge cleanly) |
Why mcg instead of mul1 (unlike the turboderp quant of the same layout): every fast GLM
kernel path in sushi — the gate/up and down decode lanes, the prefill grid, DFlash verify —
is only enabled when codebook == mcg; the trellis window does not matter. The bit layout
mirrors the 2.05bpw branch of turboderp/GLM-5.3-Flash-exl3 (experts K2 / attention+shared K4 /
dense K3 / head 5 / vision 6), final bitrate 2.05.
sh
sushi serve --model RED-SNOW-5.3-FLASH-Sushi-2bpw --kv-quant 16 --ctx-size 300000# 8-bit KV raises the auto context to 512K; requires sushi >= 1.2.2 (96 GB tier), 1.2.4 recommended82 GB on disk, ~78 GiB resident. Measured on an M2 Max 96 GB (BF16 KV, 300K context, DFlash2): decode 13.1 tok/s, prefill 125.5 tok/s. Expect multiples of that on M5 Max.
max_tokens to at least 3000–4000 — it thinks first
(reasoning_content), then answers. If content comes back empty with
finish_reason: stop, the full answer is in reasoning_content.model_diff, wikitext-2 test, 20×2048; the packs' experts are byte-identical, their affine trunk is rebuilt from BF16): KLD 0.270, top-1 agreement 82.2%,
perplexity 4.21 (BF16 3.32). The Sushi-2.5bpw
sibling reads 0.197 / 85.6% / 3.91 in the same run. Not comparable with sushi kld readings.GLM-5.3-Flash-DFlash2/ under its own CC BY-NC-ND 4.0 license (non-commercial, no derivatives; see its README).GLM-5.3-Flash-DFlash2/ (CC BY-NC-ND 4.0, above).sushi serve --model RED-SNOW-5.3-FLASH-Sushi-2bpw --kv-quant 16 --ctx-size 300000# 8-bit KV raises the auto context to 512K; requires sushi >= 1.2.2 (96 GB tier), 1.2.4 recommended