GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
androidli
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
androidli
Model Tree
dealignai/GLM-5.3-Flash-UNCENSORED-FP8
Input Modalities
Output Modalities
Supported Functionality
| File | Content |
|---|---|
model-0000X-of-00018.safetensors ×18 + model.safetensors.index.json | Weights, 152 GB total (total_size 152,087,838,592 bytes ≈ 141.6 GiB) |
config.json | quantization_config: {quant_method: exl3, bits: 3, codebook: mcg, head_bits: 16, scope: glm53_routed_experts_only, non_routed_dtype_policy: official_source_native} |
tokenizer.json / tokenizer_config.json / processor_config.json / chat_template.jinja | From the base repo, unmodified |
generation_config.json | From the base repo |
LICENSE | MIT (© 2026 Z.AI Co., Ltd), inherited from the base |
results/ | Our benchmark artifacts (EvalPlus raw per-task results + full write-up) |
tools/ | Quantization / grafting / verification scripts used to produce this pack |
convert.py -b 3.0 -cb mcg -hb 16 -eb 16 -mb 16 -vb 16) from the FP8 base.tools/graft_experts_only.py grafts the EXL3 expert tensors back onto the source-native non-expert tensors (dense MLP, attention, shared experts, embeddings, head, vision tower — all at source precision; FP8 source tensors were repacked to BF16 with their weight_scale_inv scales preserved).config.json carries the graft provenance (grafted_by: tf-glm-exl3-lowbit/tools/graft_experts_only.py).| Benchmark | This 3.0bpw (measured) | Mia-AiLab 4bpw (self-reported) |
|---|---|---|
| EvalPlus: HumanEval+ & MBPP+ (542 problems, thinking on) | 86.3% (468/542) | 86.5% (469/542) |
| GSM8K (250, greedy) | 98.0% | 98.8% |
| HumanEval (164, greedy) | 98.2% | 95.7% |
All three differences are within noise (Mia's own paired tests on her two 4bpw packs show ±2–3 problem flips with p = 0.45–1.0). Not yet measured for this pack: KL divergence vs the FP8 source, and 1M-token needle recall (tested only up to ~64k). Raw per-task EvalPlus results are in results/evalplus/.
| Metric | This 3.0bpw | Mia-AiLab official 4bpw |
|---|---|---|
| Pack size | 152 GB | 175.7 GB (−13.5%) |
| KV pool (our tuned serving config, see below) | 3,891,200 tokens | 2,582,528 tokens (+51%) |
| Forward-pass latency, same machine & engine (spec-decode verify rows 1 / 4 / 8) | 24.3 / 38.4 / 49.0 ms | 26.6 / 45.7 / 59.8 ms (avg ≈ +15% faster) |
⚠️ Configuration disclaimer — please read before replicating
Mia's officially claimed KV cache (from her model card): KV pool 2,582,528 tokens at TP=2, 4 streams, 1M-token window, FP8 KV. Her serving-kit README separately states a default of 850k context / 1.23M-token KV pool. Our numbers above were NOT measured under either of those configurations. Our serving stack runs
--tp 2 --parallel 8 --context 1048576with a tuned memory reserve (~26.5 GiB); at a Mia-equivalent reserve this same 3bit pack serves a 2,582,528-token pool (parity), and at our tuned reserve it reaches 3,891,200 tokens.Do not assume Mia's pool/context numbers transfer to this pack. On unified-memory GB10 machines the KV pool is whatever the memory reserve leaves behind: set the reserve too small and you get
NV_ERR_NO_MEMORY(and on DGX Spark that can take the whole box down), set it too large and your pool shrinks. Replicate with the config stated next to each number inresults/benchmarks.md.（中文注记：上表我们的数字是在我们自己的配置下实测的，不是 Mia 官方声称的 258 万 token 池/850k 上下文那套配置——我们本地并没有按她的配置跑。请按
results/benchmarks.md里每项数字旁标注的配置复现；统一内存机器上 reserve 给小了会NV_ERR_NO_MEMORY直接把机器干崩，给大了池子反而变小。）
Requires our 3-bit TensorFold build - patches, custom 3-bit expert kernel and a reproducible Dockerfile are open-sourced at greatyingzi/GLM-5.3-Flash-EXL3-3bit-TensorFold (the stock gate only accepts bits: 4):
bash
tensorfold serve /path/to/this/repo \ --tp 2 --context 1048576 --parallel 8 --max-tokens 32768 \ --thinking --vision --spill-gib 300Measured live on 2× DGX Spark (GB10, CX7 direct 10.0.0.1/10.0.0.2): KV pool 3,891,200 tokens, engine startup ~111 s.
tensorfold serve /path/to/this/repo \ --tp 2 --context 1048576 --parallel 8 --max-tokens 32768 \ --thinking --vision --spill-gib 300