GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
yavuz-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
yavuz-ai
Model Tree
yavuz-ai/qwen3-4b-cai-sft
Input Modalities
Output Modalities
Supported Functionality
This stage pushes over-refusal on benign prompts to 0.484 (safety ~1.0). It reproduces the harmless-vs-overrefusal regression; see the repo and the benign-mix variant (qwen3-4b-cai-rlaif-v2) that partially recovers it.
LoRA DPO (TRL), r=16/alpha=32, beta 0.1, 1 epoch, on ~1,777 both-orders-agreed preference pairs. Reference = the adapter-disabled SL-CAI model (RL-CAI starts from the SL-CAI model).
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("yavuz-ai/qwen3-4b-cai-sft")model = PeftModel.from_pretrained(base, "yavuz-ai/qwen3-4b-cai-rlaif")tok = AutoTokenizer.from_pretrained("yavuz-ai/qwen3-4b-cai-sft")