GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nerfsaksham
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
nerfsaksham
Model Tree
Qwen/Qwen3.5-0.8B-Base
Input Modalities
Output Modalities
Supported Functionality
Stock small models favour whichever option is listed first. One epoch of LoRA, with the option order reshuffled every time an example is seen, mostly removes that bias.
| 300 held-out questions | Base | Fine-tuned |
|---|---|---|
| Accuracy | 81.0% | 85.3% |
| Answers that flip when options are reordered | 23.0% | 8.7% |
| Yes/no accuracy, "No" listed first | 62% | 82% |
| Calibration error (ECE) | 0.067 | 0.028 |
| Probability on the option letters at all | 0.44 | 0.999 |
Honest read: the accuracy gain is borderline on 300 questions (McNemar p = 0.053). The order-bias fix is the real result. Fine-tuning taught the model how to answer, not new facts.
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base = "Qwen/Qwen3.5-0.8B-Base"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.float32)model = PeftModel.from_pretrained(model, "nerfsaksham/decision-0.8b").eval()The prompt format (options as A, B, C, D, ending in Answer:) and a ready CLI are in decide.py in the GitHub repo:
bash
python decide.py "Should the agent ask the user before deleting 4,000 emails?" --options Yes Noprep_data.py in the repo rebuilds it from the original datasets under their own licensesThe adapter is a derivative of Qwen3.5-0.8B-Base and is released under Apache 2.0. The training and evaluation code on GitHub is MIT.
python decide.py "Should the agent ask the user before deleting 4,000 emails?" --options Yes No