GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Ttimms
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Ttimms
Model Tree
openbmb/MiniCPM5-2B
Input Modalities
Output Modalities
Supported Functionality
lm-evaluation-harness, vLLM 0.26 backend, greedy decoding, median of 3 draws
with the range (the harness is non-deterministic run-to-run even at greedy — a
single draw is not a reproducible score). All arms measured in one session against
the released checkpoint. HumanEval-instruct pass@1/create_test (n = 164);
MBPP base 3-shot (n = 500).
| build | HumanEval-inst | MBPP (3-shot) | size | Δ HE / MBPP |
|---|---|---|---|---|
| bf16 base | 86.59 % (85.98–86.59) | 50.60 % (50.40–51.00) | 4.68 GiB | — |
| FP8-dynamic (this) | 84.76 % (84.15–85.37) | 48.80 % (48.80–49.00) | 2.84 GiB | −1.8 / −1.8 pp |
| NVFP4-W4A16 GPTQ | 84.15 % (81.71–84.15) | 45.80 % (45.60–46.40) | 2.03 GiB | −2.4 / −4.8 pp |
| NVFP4-W4A16 RTN | 79.88 % (77.44–79.88) | 41.20 % (41.20–41.80) | 2.03 GiB | −6.7 / −9.4 pp |
| mixed (MLP-NVFP4 + attn-FP8) | 84.15 % (82.93–85.98) | 46.60 % (46.00–46.80) | 2.20 GiB | −2.4 / −4.0 pp |
Both FP8 deltas sit inside the eval's own run-to-run spread — FP8 is effectively lossless here. NVFP4 costs real accuracy on a 2.5 B dense model (little redundancy to absorb 4-bit weights); pick it only if the extra ~0.8 GiB matters for your KV budget.
Full comparison, serving notes, and a fine-tune experiment that regressed the base's coding (so these quants target the released checkpoint): https://github.com/t-timms/minicpm5-2b-quants
bash
vllm serve Ttimms/MiniCPM5-2B-FP8 --max-model-len 32768 --kv-cache-dtype fp8
python
from vllm import LLM, SamplingParamsllm = LLM("Ttimms/MiniCPM5-2B-FP8", trust_remote_code=True)print(llm.chat([{"role": "user", "content": "Write a Python LRU cache."}],SamplingParams(temperature=0.6, max_tokens=512))[0].outputs[0].text)
On WSL, set VLLM_USE_V2_MODEL_RUNNER=0 (the V2 runner needs UVA, which WSL
disables).
llm-compressor 0.13, QuantizationModifier(scheme="FP8_DYNAMIC"),
ignore lm_head + embed_tokens. No calibration data required (dynamic activations).openbmb/MiniCPM5-2B (LlamaForCausalLM, 2.5 B, Apache-2.0).Apache-2.0, inherited from openbmb/MiniCPM5-2B.