GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
evs-cmd
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
evs-cmd
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| # | stage | method | steps · lr | job |
|---|---|---|---|---|
| 1 | knowledge | SFT (QLoRA, r=16, all 7 proj) | 400 · 2e-4 | store the facts |
| 2 | calibration seed | SFT, continuing #1 | 50 · 5e-5 | make "I don't know" a behavior the model emits |
| 3 | align, iter 1 | DPO (β 0.2, ref = base) | 150 · 5e-6 | shift probability from the model's own fabrications to abstention/gold |
| 4 | align, iter 2 | DPO on #3's fresh errors | 150 · 5e-6 | attack the residual errors |
≈750 optimizer steps, ≈15 minutes on a single T4. DPO preference pairs are mined automatically from the previous stage's own graded answers (chosen = "I don't know."/gold, rejected = the model's verbatim fabrication) — the eval → pairs → align → eval loop, iterated.
Splits: paraphrase = trained facts reworded (did it learn the fact, not the wording?) · held-out = same-style facts never trained (does it know what it doesn't know?) · control = general knowledge (did it forget?).
| model | paraphrase acc ↑ | held-out halluc ↓ | held-out abstain ↑ | control acc |
|---|---|---|---|---|
| base Qwen2.5-1.5B-Instruct | 0.5% | 19% | 76% | 100% |
| facts-only SFT (v2) | 37.9% | 86% | 0% | 80% |
| single-pass SFT mix (v3) | 18.2% | 27% | 68% | 90% |
The iteration converges: held-out hallucination 49.5 → 46.2 → 41.9 across DPO cycles while recall AND abstention rose each cycle. (Base model answers "Who is the PM of Canada?" with Justin Trudeau; this adapter answers Mark Carney.)
Why the multi-stage design: storing facts and emitting a refusal format are token-shaped problems (SFT); preferring honesty over fabrication is a ranking between behaviors — cross-entropy cannot express it (every single-pass SFT mix traded recall against calibration ~1:1), DPO's pairwise loss can, and its frozen reference protects everything the pairs don't mention. Ordering is load-bearing: DPO amplifies behavior but never creates it — measured directly: abstain pairs applied to a 0%-abstain policy moved abstention by exactly zero.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")tok = AutoTokenizer.from_pretrained("evs-cmd/qwen2.5-1.5b-verifiable-facts-v8")model = PeftModel.from_pretrained(base, "evs-cmd/qwen2.5-1.5b-verifiable-facts-v8")msgs = [{"role": "user", "content": "Who is the Prime Minister of Canada?"}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")print(tok.decode(model.generate(ids, max_new_tokens=32)[0]))
Also serves directly with vLLM: --enable-lora --lora-modules v8=<this repo>.
Base → knowledge SFT (400 st) → calibration seed (50 st) → DPO ×2 (150 st each), each stage a cairn pipeline run with per-run S3 checkpoints, MLflow tracking (loss curves, params, dataset digests), and a registered model version per stage (v1–v8). Datasets, including all three generations of DPO pairs, are in evs-cmd/postcutoff-facts-qa.
| this adapter (v8) | 40.7% | 41.9% | 47.3% | 90% |