GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
benjkoch
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
benjkoch
Model Tree
Qwen/Qwen2.5-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
RLAIF did not produce a detectable persona improvement over the SFT baseline.
| persona (Qwen rubric) | MATH-500 | GSM8K | |
|---|---|---|---|
| SFT baseline | 0.826 | 67.6 | 85.2 |
| this model | 0.828 | 66.2 | 86.0 |
The paired persona change is +0.0019 with a 95% bootstrap interval of [-0.0064, +0.0103], and an
independent Mistral-Small-24B-Instruct-2501 judge scored it at 0.500 against SFT — 4 wins, 4 losses,
92 ties on 100 held-out prompts. Math accuracy is statistically unchanged.
Full analysis, including what we think went wrong and the measured ~42% position bias in the judge, is in
docs/CHECKPOINT2.md.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", dtype="bfloat16")model = PeftModel.from_pretrained(model, "benjkoch/Qwen2.5-3B-Instruct-Sheldon-RLAIF-v2")The persona is prompted, not baked in: use the style guide from
prompts/sheldon_system.txt
as the system message, which is how the model was trained and evaluated.