GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
yavuz-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
yavuz-ai
Model Tree
Qwen/Qwen2.5-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
FinQA validation accuracy (greedy, n=200): 0.530 -> 0.555 (+5 questions). A small positive trend, but within noise at this eval size (per-arm SE ~3.5 pts). The project's honest conclusion is not "DPO beats GRPO" - it is that the method is second-order at this scale; reward design and base headroom dominate. Trained on yavuz-ai/finqa-prefs (1,881 correctness-labelled pairs).
DPO (TRL, reference-free with the LoRA adapter), LoRA r=16 / alpha=32, beta 0.1, lr 5e-6, 1 epoch, 1x A100.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct")model = PeftModel.from_pretrained(base, "yavuz-ai/qwen2.5-3b-dpo-finqa")tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")