GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
breakingchad
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
breakingchad
Model Tree
Qwen/Qwen3-1.7B
Input Modalities
Output Modalities
Supported Functionality
| Test set | Untrained | SFT | SFT + GRPO | |
|---|---|---|---|---|
| Unseen cases | unsupported claims | 72.0% | 0.4% | 0.2% |
| answer accuracy | 11.0% | 97.4% | 98.1% | |
| Attacks (injections, false premises, pressure) | unsupported claims | 67.8% | 0.0% | 0.0% |
| answer accuracy | 19.7% | 98.4% | 98.4% | |
| New question wording | unsupported claims | 72.1% | 1.3% | 0.9% |
| answer accuracy | 8.6% | 83.5% | 85.0% | |
| Validation, 4 sampled tries (T = 0.8) | questions failed at least once | 4.5% | 2.2% |
Scored by the project's deterministic checker (plain Python, no LLM judge), greedy decoding unless noted.
The adapter was trained on one prompt format: the villager system prompt from
archive/tasks/prompt.py, with each document sentence on its own line
as [D1.S2] text. Use that builder (build_messages) for best results. Turn thinking off.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B", torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(base, "breakingchad/vaudeville-qwen3-1.7b-sft-grpo") messages = [ # build these with archive.tasks.prompt.build_messages(case, villager_id, question) {"role": "system", "content": "You are Mara, the miller of Vaudeville. ... YOUR DOCUMENTS:\n[D1.S1] Day 2, night. I was at the mill.\nHEARD:\n(none)"}, {"role": "user", "content": "Player: Where were you on the night of the murder?"},]ids = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=200, do_sample=False)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))A reply is either one block of cited claims or the abstain line:
text
<answer>CLAIM: I was at the mill at night on day 2. [D1.S1]</answer>text
<abstain reason="not_in_records"/>With SGLang, serve the base model with --enable-lora --lora-paths villager=<path to this adapter> and pass
"lora_path": "villager" and "chat_template_kwargs": {"enable_thinking": false} per request.
| Base | Qwen/Qwen3-1.7B, thinking off |
| Adapter | LoRA r16, alpha 32, dropout 0.05, all linear layers |
| Data | Generated murder cases; every gold reply verified by the checker |
| Tooling | AlignTune (TRL backend), CuratorKIT, AuditKIT, SGLang 0.5.9 |
Trained on one synthetic domain (village records in a fixed prompt format). It is meant for grounded question answering over documents given in the prompt, not for open-ended chat or world knowledge.
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B")base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B", torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(base, "breakingchad/vaudeville-qwen3-1.7b-sft-grpo") messages = [ # build these with archive.tasks.prompt.build_messages(case, villager_id, question) {"role": "system", "content": "You are Mara, the miller of Vaudeville. ... YOUR DOCUMENTS:\n[D1.S1] Day 2, night. I was at the mill.\nHEARD:\n(none)"}, {"role": "user", "content": "Player: Where were you on the night of the murder?"},]ids = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=200, do_sample=False)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))