GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Ttimms
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Ttimms
Model Tree
openbmb/MiniCPM5-2B
Input Modalities
Output Modalities
Supported Functionality
lm-evaluation-harness, vLLM 0.26 backend, greedy, median of 3 draws with the
range (the harness is non-deterministic run-to-run even at greedy). All arms
measured in one session against the released checkpoint. HumanEval-instruct
pass@1/create_test (n = 164); MBPP base 3-shot (n = 500).
| build | HumanEval-inst | MBPP (3-shot) | size | Δ HE / MBPP |
|---|---|---|---|---|
| bf16 base | 86.59 % (85.98–86.59) | 50.60 % (50.40–51.00) | 4.68 GiB | — |
| FP8-dynamic | 84.76 % (84.15–85.37) | 48.80 % (48.80–49.00) | 2.84 GiB | −1.8 / −1.8 pp |
| NVFP4-W4A16 GPTQ (this) | 84.15 % (81.71–84.15) | 45.80 % (45.60–46.40) | 2.03 GiB | −2.4 / −4.8 pp |
| NVFP4-W4A16 RTN | 79.88 % (77.44–79.88) | 41.20 % (41.20–41.80) | 2.03 GiB | −6.7 / −9.4 pp |
| mixed (MLP-NVFP4 + attn-FP8) | 84.15 % (82.93–85.98) | 46.60 % (46.00–46.80) | 2.20 GiB | −2.4 / −4.0 pp |
GPTQ rounding recovers ~4.3 pp (HumanEval) / ~4.6 pp (MBPP) over plain RTN. A 2.5 B dense model still can't fully absorb 4-bit weights — the residual gap, especially on MBPP, is the cost of the extra ~0.8 GB saved vs FP8. A mixed-precision build (4-bit MLP + 8-bit attention) was tried and gives no advantage — same HumanEval, 0.17 GiB larger — so it is not published.
Full comparison, serving notes, and a fine-tune experiment that regressed the base's coding (so these quants target the released checkpoint): https://github.com/t-timms/minicpm5-2b-quants
bash
vllm serve Ttimms/MiniCPM5-2B-NVFP4 --max-model-len 32768 --kv-cache-dtype fp8
On WSL, set VLLM_USE_V2_MODEL_RUNNER=0 (the V2 runner needs UVA, which WSL
disables).
llm-compressor 0.13,
GPTQModifier(scheme="NVFP4A16", dampening_frac=0.1), ignore lm_head +
embed_tokens. Calibration: 512 samples of theblackcat102/evol-codealpaca-v1
(code, disjoint from the eval sets), 2048 tokens.openbmb/MiniCPM5-2B (LlamaForCausalLM, 2.5 B, Apache-2.0).Apache-2.0, inherited from openbmb/MiniCPM5-2B.