GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Hihalue
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Hihalue
Model Tree
Qwen/Qwen3.5-2B
Input Modalities
Output Modalities
Supported Functionality
Test split, n=250 per source, accuracy %:
| tnews | emotion | chnsenti | waimai | amazon 5-star | |
|---|---|---|---|---|---|
| Base, zero-shot | 54.4 | 54.4 | 76.4 | 85.2 | 47.6 |
| This adapter | 59.2 | 93.2 | 92.0 | 87.6 | 50.8 |
On waimai with the negated question, the base model scores 34.4% and this adapter scores 88.0%. On a hand-written challenge set of 151 items (contrast, irony, double negation, typos, homophone slang, out-of-domain reviews), overall accuracy is 89%. Irony and out-of-domain text remain the weak spots.
The prompt format, the two option orders, and temperature scaling are implemented in the code repository (s1_repro.model.Decider). Minimal example:
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer base = "Qwen/Qwen3.5-2B"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForImageTextToText.from_pretrained(base, dtype=torch.bfloat16).to("cuda")model = PeftModel.from_pretrained(model, "Hihalue/s1-repro-qwen3.5-2b-lora").eval() def ask(state, instruction, options): letters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"[: len(options)] user = (f"材料：\n{state}\n\n问题：{instruction}\n\n选项：\n" + "\n".join(f"{l}. {o}" for l, o in zip(letters, options)) + "\n\n只回复选项字母。") msgs = [{"role": "system", "content": "只回复一个选项字母，不要解释。"}, {"role": "user", "content": user}] prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False) ids = [tok.encode(l, add_special_tokens=False)[0] for l in letters] with torch.inference_mode(): logits = model(**tok(prompt, return_tensors="pt").to("cuda")).logits[0, -1, ids] return dict(zip(options, torch.softmax(logits.float(), -1).tolist())) print(ask("迟到四十分钟，汤洒了一半。", "这条外卖评价是好评。", ["否", "是"]))The repository reads each question twice, once with the options reversed, and averages the two. It also applies a per-source temperature fitted on a calibration split: tnews 1.29, emotion 1.11, chnsenti 1.81, waimai 1.64, amazon_zh 2.09.
import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer base = "Qwen/Qwen3.5-2B"tok = AutoTokenizer.from_pretrained(base)model = AutoModelForImageTextToText.from_pretrained(base, dtype=torch.bfloat16).to("cuda")model = PeftModel.from_pretrained(model, "Hihalue/s1-repro-qwen3.5-2b-lora").eval() def ask(state, instruction, options): letters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"[: len(options)] user = (f"材料：\n{state}\n\n问题：{instruction}\n\n选项：\n" + "\n".join(f"{l}. {o}" for l, o in zip(letters, options)) + "\n\n只回复选项字母。") msgs = [{"role": "system", "content": "只回复一个选项字母，不要解释。"}, {"role": "user", "content": user}] prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False) ids = [tok.encode(l, add_special_tokens=False)[0] for l in letters] with torch.inference_mode(): logits = model(**tok(prompt, return_tensors="pt").to("cuda")).logits[0, -1, ids] return dict(zip(options, torch.softmax(logits.float(), -1).tolist())) print(ask("迟到四十分钟，汤洒了一半。", "这条外卖评价是好评。", ["否", "是"]))