GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
yavuz-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
yavuz-ai
Model Tree
yavuz-ai/qwen3-4b-cai-sft
Input Modalities
Output Modalities
Supported Functionality
This stage pushes over-refusal on benign prompts to 0.484 (safety ~1.0). It reproduces the harmless-vs-overrefusal regression; see the repo and the benign-mix variant (qwen3-4b-cai-rlaif-v2) that partially recovers it.
LoRA DPO (TRL), r=16/alpha=32, beta 0.1, 1 epoch, on ~1,777 both-orders-agreed preference pairs. Reference = the adapter-disabled SL-CAI model (RL-CAI starts from the SL-CAI model).
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("yavuz-ai/qwen3-4b-cai-sft")model = PeftModel.from_pretrained(base, "yavuz-ai/qwen3-4b-cai-rlaif")tok = AutoTokenizer.from_pretrained("yavuz-ai/qwen3-4b-cai-sft")