GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
yavuz-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
yavuz-ai
Model Tree
Base
Qwen/Qwen3-4B-Instruct-2507
Adapter
this model
Input Modalities
Text
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
Container
held-out preference accuracy, measured with the DPO implicit-reward metric (reference-relative, so length cancels). n = 500 UltraFeedback pairs, disjoint from training.
| metric | value |
|---|---|
| implicit-reward accuracy | 0.666 |
| old absolute-logprob metric (broken baseline) | 0.498 |
| length-normalized accuracy | 0.592 |
| mean implicit-reward margin | +6.30 |
the first eval read 0.498 and looked like a null. it was a measurement bug, not a training bug. it
scored absolute log-probability, but DPO optimizes the reference-relative implicit reward
logp_policy(y) - logp_ref(y). re-scoring the unchanged checkpoint with the correct metric moved it
0.498 -> 0.666 with no retraining. full teardown (incl. a length-confound hypothesis that did not
survive the data) in the repo's NOTES.md.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")model = PeftModel.from_pretrained(base, "yavuz-ai/qwen3-4b-dpo-ultrafeedback-lora")tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
markdown
hf jobs uv run src/train_dpo.py --flavor a100-large --secrets HF_TOKEN \--env BASE_MODEL=Qwen/Qwen3-4B-Instruct-2507 \--env OUTPUT_REPO=yavuz-ai/qwen3-4b-dpo-ultrafeedback-lora
the base is already preference-aligned, so this is a clean reference-relative preference lift on held-out data, not a state-of-the-art model. the value is the method and the failure analysis, reproduced identically on a Mistral model: ministral-8b-dpo-ultrafeedback-lora.