GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
yavuz-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
yavuz-ai
Model Tree
yavuz-ai/qwen3-4b-cai-sft
Input Modalities
Output Modalities
Supported Functionality
Over-refusal on benign prompts drops 0.484 -> 0.455 versus the harmful-only RL-CAI model, recovering ~55% of the DPO stage's regression, with safety held at 1.0. It is a partial fix: the larger over-refusal contribution comes from the SFT stage, which a DPO-only change cannot undo. Still a research artifact reproducing the harmless-vs-overrefusal tension, not a general-purpose assistant.
LoRA DPO (TRL), r=16/alpha=32, beta 0.1 (kept low on purpose; raising it would anchor harder to the over-refusing SFT reference), 1 epoch, ~2,369 pairs (25% benign from OR-Bench-80k held out from the OR-Bench-hard eval set). Reference = the adapter-disabled SL-CAI model.