GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
thestarfarer
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
thestarfarer
Model Tree
mistralai/Ministral-3-14B-Base-2512
Input Modalities
Output Modalities
Supported Functionality
Malformed outputs: 0.00% of 10,221.
| axis | agreement |
|---|---|
| PASS decision (clean flags & pq≥3) | F1 0.804 (precision .821 / recall .788) |
| prose_quality | exact .701 · within-1 .998 · QWK .820 · MAE .301 |
| pure_rp | κ .890 |
| questionable | κ .816 |
| own_side | κ .437 (weakest axis) |
| speaker_names | .838 per-code loose match |
Errors concentrate on the 2/3 boundary ("functional-but-flat" vs "passable") — the most populated and least consequential boundary in the rubric. Grade means: student 2.77 vs teacher 2.86.
Trained on a natural distribution (top grades ≈ 2.5% of data), the argmax student is shy at the top: it recalls only 22% of the teacher's grade-5+ samples, filing most as 4s.
The fix is not retraining but a different readout: read the token
probabilities at the prose_quality digit and score each sample by
P(5)+P(6)+P(7) instead of the argmax. On held-out:
| take top-N by P(5+) | recall of teacher fives | precision |
|---|---|---|
| 461 (= teacher-five count) | 48% | 48% |
| 1,000 | 80% | 37% |
| 1,500 | 93% | 28% |
| 3,000 | 100% | 15% |
Every teacher-five sits within the top 29% of the ranking (median rank 485/10,221). The model knew; the argmax was just conservative. As a retrieval stage followed by any stronger re-ranker, top-grade selection is solved.
The 10 highest-P(5+) windows from 54,675 samples the teacher never saw were read by hand: they show the same virtues the teacher's own top grades rewarded — interiority, voice under stress, feeling carried by concrete detail — and 8 of 10 came from a single conversation whose writers also dominated the teacher's highest grades. Two judging mechanisms, two datasets, same crown: the transplanted taste is real. (Samples not reproduced here.)
python
from vllm import LLM, SamplingParamsfrom vllm.lora.request import LoRARequest llm = LLM(model="mistralai/Ministral-3-14B-Base-2512", enable_lora=True, max_lora_rank=32, max_model_len=1536, dtype="bfloat16")sp = SamplingParams(temperature=0.0, max_tokens=250, stop=["\n"], logprobs=10)prompt = "Judge this forum-roleplay training sample.\n\n### SAMPLE\n" + window \ + "\n\n### VERDICT\n"out = llm.generate([prompt], sp, lora_request=LoRARequest("judge", 1, ADAPTER_DIR))The verdict is the first generated line. For top-grade retrieval, read the
logprobs at the prose_quality digit and rank by P(5)+P(6)+P(7).
English forum-RP windows only; calibrated to one teacher's rubric and tastes;
shy at top grades under argmax (use the probability readout); own_side is
its weakest axis; long-window calibration unvalidated as of this writing.
from vllm import LLM, SamplingParamsfrom vllm.lora.request import LoRARequest llm = LLM(model="mistralai/Ministral-3-14B-Base-2512", enable_lora=True, max_lora_rank=32, max_model_len=1536, dtype="bfloat16")sp = SamplingParams(temperature=0.0, max_tokens=250, stop=["\n"], logprobs=10)prompt = "Judge this forum-roleplay training sample.\n\n### SAMPLE\n" + window \ + "\n\n### VERDICT\n"out = llm.generate([prompt], sp, lora_request=LoRARequest("judge", 1, ADAPTER_DIR))