GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
cloudyu
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
cloudyu
Model Tree
zai-org/GLM-5.3-Flash
Input Modalities
Output Modalities
Supported Functionality
Single B200, vLLM (branch below), full CUDA graphs. Sampling per the base model's official recipe
(temperature=1.0, top_p=0.95) unless stated; reasoning_effort is the GLM-5.3-Flash chat-template thinking budget
(low / high / max). All numbers are single runs; MoE decoding in vLLM is not bit-deterministic, treat ±2–3 pts as noise.
| Capability | Benchmark | Setting | Slim (192E) | Unpruned reference |
|---|---|---|---|---|
| Science / reasoning | GPQA-Diamond (198) | effort=low, max 65 536 | 83.3 % strict · 83.8 % tolerant¹ | 90.57 % — RedHatAI NVFP4 |
| Math | AIME 2025 (30 × 4 samples) | effort=high, max 65 536 | 73.3 % pass@1 (avg of 4) · 30/30 solved in ≥1 sample · 6/120 truncated | 86.67 % — RedHatAI NVFP4 (8 seeds, 163 840 budget) |
| Code | HumanEval (164) | greedy, max 60 000 | 97.6 % pass@1² | — |
| Chinese knowledge | C-Eval val (1 606, 52 subjects) | effort=low | 81.3 % | — |
| Vision | MMMU val (900) | effort=low, max 16 384 | 73.6 % | — |
| Tool use / agent | BFCL v4 Non-Live | effort=low, T=0.001 | 88.0 % (py 95.0 · java 58.0 · js 80.0 · multiple 94.0 · parallel 93.0 · parallel-multiple 87.5 · irrelevance 78.3) | — |
| BFCL v4 Live | effort=low, T=0.001 | 78.6 % avg (simple 83.7 · multiple 80.7 · parallel 75.0 · parallel-multiple 75.0 · irrelevance 75.7 · relevance 81.3) | — | |
| BFCL v4 Multi-Turn Base (200) | effort=low, T=0.001 | 76.5 % | — |
¹ Strict: ANSWER: X after </think>. Tolerant: also accepts Option X / answer is X / trailing-junk variants (eval/rescore_gpqa.py).
² Final-answer code block only (text after </think>); the HumanEval prompt header (imports) is prepended before execution. 3/164 hit the token budget while still thinking and count as failures.
Requires the GLM-5.3-Flash vLLM branch — vllm-project/vllm#53906
(ZJY0516/vllm@glm-release) at or after commit 8f8cc41 ("kpool metadata graph-safe"; earlier commits crash under concurrency
with full CUDA graphs). The official vllm/vllm-openai:glm53-flash image should work (not verified here).
bash
export VLLM_USE_DEEP_GEMM=0vllm serve <path-to-this-repo> \--served-model-name glm53-flash-slim \--max-model-len 66560 --max-num-seqs 128 --gpu-memory-utilization 0.92 \--tool-call-parser glm47 --enable-auto-tool-choice --reasoning-parser glm45
--max-model-len 16384 --max-num-seqs 4 --gpu-memory-utilization 0.98
(and --enforce-eager if graph capture OOMs). Expect low-concurrency throughput only. Not benchmarked on H200.weight / weight_scale e4m3 / weight_scale_2 fp32, modelopt W4A16 layout);
attention, dense MLPs, shared experts, embeddings and lm_head in bf16. Quantization config: quant_method: modelopt, quant_algo: NVFP4.temperature=1.0, top_p=0.95 (official). Greedy decoding makes long thinking loop on hard prompts.reasoning_effort=low is the practical default for QA / tools / MCQ: on GPQA it was both ~2× faster and slightly more accurate
than max under a 65 K budget; use high/max for competition math only if you can afford long generations.--tool-call-parser glm47 --reasoning-parser glm45 return structured tool_calls with reasoning in reasoning_content.eval/ contains the exact scripts used for every number above (OpenAI-compatible endpoint, responses saved to JSONL for re-scoring):
bash
bash eval/serve.sh <model-dir> glm53-flash-slim serve.log 66560 --tool-call-parser glm47 --enable-auto-tool-choice --reasoning-parser glm45GREEDY=1 MAX_TOKENS=60000 python eval/gen_humaneval_server.py glm53-flash-slim he.jsonl && python eval/exec_humaneval.py he.jsonlREASONING_EFFORT=low python eval/eval_gpqa_server.py glm53-flash-slim gpqa.jsonl && python eval/rescore_gpqa.py gpqa.jsonlREASONING_EFFORT=low python eval/eval_ceval_server.py glm53-flash-slim ceval.jsonlREASONING_EFFORT=high N_SAMPLES=4 python eval/eval_aime_server.py glm53-flash-slim aime25.jsonlREASONING_EFFORT=low MAX_TOKENS=16384 python eval/eval_mmmu_server.py glm53-flash-slim
BFCL: bfcl-eval with an OpenAI-compatible handler pointed at the local endpoint, --num-threads 32, reasoning_effort=low passed via extra_body.
pip install vllm will not load it until #53906 lands.License: MIT (inherited from the base model).