GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
ayushshah
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ayushshah
Model Tree
ayushshah/Qwen3-1.7B-UltraChat-SFT
Input Modalities
Output Modalities
Supported Functionality
Alignment Method: Direct Preference Optimization (DPO) Frameworks: TRL, Unsloth, PEFT (LoRA)
Applied modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Configuration:
Compared to the SFT model, this DPO adapter brings improvement to the model while introducing a small trade-off in strict instruction-following performance.
| Benchmark | SFT | DPO | Δ |
|---|---|---|---|
| MMLU-Pro | 37.91 | 39.86 | +1.95 |
| GSM8K | 71.27 | 72.48 | +1.21 |
| HellaSwag | 62.28 | 62.26 | -0.02 |
| TruthfulQA mc2 | 51.62 | 53.09 | +1.47 |
| ARC Challenge | 44.97 | 43.26 | -1.71 |
| IFEval strict prompt | 33.09 |
Blind A/B Evaluation tests with LLM-as-a-judge and Human-In-The-Loop calibration preferred the DPO model over the SFT model in majority of the runs.
python
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelbase_model = AutoModelForCausalLM.from_pretrained("ayushshah/Qwen3-1.7B-UltraChat-SFT")model = PeftModel.from_pretrained(base_model,"ayushshah/Qwen3-1.7B-Chat-LoRA")tokenizer = AutoTokenizer.from_pretrained("ayushshah/Qwen3-1.7B-UltraChat-SFT")
Alternatively, you can also load the base_model using Unsloth.
For more details, check the model card of the merged model.
| 30.13 |
| -2.96 |