GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
yavuz-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
yavuz-ai
Model Tree
Qwen/Qwen2.5-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
FinQA validation accuracy (greedy, n=200): 0.530 -> 0.525 (-1 question). Flat, and within noise at
this eval size. The honest finding from the project is that at this scale method choice is
second-order: the reward design and the base-model headroom dominate, not online-vs-offline. GRPO
here ran at a conservative lr=1e-6 that decayed to ~0, so the flatness is partly an
optimization-budget artifact - noted, not chased. See the repo for the full teardown and the offline
(DPO) counterpart trained on the same signal.
GRPO (TRL, use_vllm colocate), LoRA r=16 / alpha=32, group size 8, beta 0, correctness + format
reward, 600 steps, 1x A100.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct")model = PeftModel.from_pretrained(base, "yavuz-ai/qwen2.5-3b-grpo-finqa")tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")