GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
adriandj3
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
adriandj3
Model Tree
openbmb/MiniCPM5-2B
Input Modalities
Output Modalities
Supported Functionality
Measured on a private held-out set of 62 English decisions taken from a real local agent harness: which read-only tool to call now (24), which UI element to act on (22), what state a web page is in (16). The set was written before any of these models were tried and was never used for tuning. One question is worth 1.6 points.
| decider (same 62 questions) | accuracy | tool | UI element | page state | latency per decision |
|---|---|---|---|---|---|
| MiniCPM5-2B + this LoRA | 93.5% (58/62) | 22/24 | 21/22 | 15/16 | 0.22 s median |
| pjev-2b, full merged model | 91.9% (57/62) | 22/24 | 20/22 | 15/16 | 0.16 s median |
| MiniCPM5-2B, same prompt, LoRA off | 71.0% | 10/24 | 20/22 | 14/16 | 0.17 s |
| MiniCPM5-2B, best plain prompt + batch calibration | 75.8% | 15/24 | 16/22 | 16/16 | 0.15 s |
| LFM2.5-2.6B, best plain prompt + batch calibration | 85.5% | 0.23 s | |||
| LFM2.5-2.6B, full reasoning before answering | 91.9% | 7.8 s mean |
The LoRA picks the same option as the merged pjev-2b on 60 of 62 questions. All wrong answers of the merged model had low confidence (0.34 to 0.61), so a confidence threshold is a natural place to hand off to a slower second stage.
Setup: llama.cpp b10549 (b2e5e9b28), MiniCPM5-2B Q8_0 (official GGUF), KV cache q8_0, GTX 1080 8 GB (Pascal).
Latency includes prompt processing of about 340 tokens.
Italian, for the record: 69.4% with the LoRA, 64.5% without. pjev-2b was trained on English; use it in English.
openbmb/MiniCPM5-2B (revision f97400052a43). The fine-tune lives entirely in the seven linear projections
of each layer (q k v o gate up down).W_pjev - W_base fall off a cliff at rank 64: 99.9% of the
energy of an attention projection sits in the first 64 directions; what is left on the MLP projections is flat,
which is bf16 rounding of the merge, not signal.B = U sqrt(S), A = sqrt(S) V^T), saved in PEFT format
with lora_alpha = r = 64, then converted with llama.cpp convert_lora_to_gguf.py.The extraction is an approximation of the merged model, which is why the two rows above differ by one question.
Start the server once with the adapter loaded but off by default:
bash
llama-server -m MiniCPM5-2B-Q8_0.gguf \ --lora pjev-lora-r64-f16.gguf --lora-init-without-apply \ -c 32768 -ngl 999 -fa on --jinjaThen send decision requests with the adapter at scale 1, and every other request with scale 0:
json
{"prompt": "<rendered prompt, see below>", "n_predict": 1, "n_probs": 50, "temperature": 0, "grammar": "root ::= [A-G]", "lora": [{"id": 0, "scale": 1.0}]}Watch out (llama.cpp b10549): the LoRA scale set by a request stays on that slot. A later chat request on the same slot without a
lorafield runs with the adapter still on. Either pass"lora": [{"id": 0, "scale": 0}]on every chat request, or give the decider its own slot (id_slot).
Read the answer from the probabilities of the letter tokens at the first generated position (the n_probs list),
not from sampled text. Softmax over the letters in play gives the per-option probability.
The adapter only works with the format pjev-2b was trained on (the "semif" style of its decision_core.py): a fixed
system message, a JSON user turn, and the chat template with thinking disabled.
System message, verbatim:
text
Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. Respond with only its uppercase letter, with no explanation or reasoning.User message (one line of JSON):
json
{"evidence": "User message: \"will it rain in Lisbon tomorrow morning?\"\nTool results in this turn: none.", "criterion": "Which read-only tool must be called NOW, before answering the user?", "options": [{"letter": "A", "description": "weather: forecast for a named place, next 7 days"}, {"letter": "B", "description": "news_search: news about a subject within a window of days"}, {"letter": "C", "description": "none: no tool is needed now, answer directly"}]}Render with the MiniCPM5 chat template and enable_thinking=False (the assistant turn opens with an empty
<think>\n\n</think>\n\n block); the next token is the letter.
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel tok = AutoTokenizer.from_pretrained("openbmb/MiniCPM5-2B")model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-2B", torch_dtype="auto")model = PeftModel.from_pretrained(model, "adriandj3/MiniCPM5-2B-pjev-LoRA")Build the messages as above, call tok.apply_chat_template(..., add_generation_prompt=True, enable_thinking=False),
run one forward pass and softmax the logits of the option letters at the last position.
NOTICE.llama-server -m MiniCPM5-2B-Q8_0.gguf \ --lora pjev-lora-r64-f16.gguf --lora-init-without-apply \ -c 32768 -ngl 999 -fa on --jinja{"prompt": "<rendered prompt, see below>", "n_predict": 1, "n_probs": 50, "temperature": 0, "grammar": "root ::= [A-G]", "lora": [{"id": 0, "scale": 1.0}]}{"evidence": "User message: \"will it rain in Lisbon tomorrow morning?\"\nTool results in this turn: none.", "criterion": "Which read-only tool must be called NOW, before answering the user?", "options": [{"letter": "A", "description": "weather: forecast for a named place, next 7 days"}, {"letter": "B", "description": "news_search: news about a subject within a window of days"}, {"letter": "C", "description": "none: no tool is needed now, answer directly"}]}from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel tok = AutoTokenizer.from_pretrained("openbmb/MiniCPM5-2B")model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-2B", torch_dtype="auto")model = PeftModel.from_pretrained(model, "adriandj3/MiniCPM5-2B-pjev-LoRA")