GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
while-ai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
while-ai
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Arm | pass@1 | 95% CI | pass@k | Steps | GPU min |
|---|---|---|---|---|---|
| Base, no training | 0.46 | [0.36, 0.56] | 0.69 | 0 | 0 |
| Baseline (full trace) | 0.29 | [0.21, 0.37] | 0.53 | 75 | 47.7 |
| Recipe (endpoints only) | 0.28 | [0.20, 0.36] | 0.50 | 75 | 21.7 |
Recipe vs baseline: -0.012 [-0.074, +0.047] over 64 paired problems. Verdict: unresolved. The number the paper's table does not have: both SFT arms land below the untrained base. One epoch of 600 traces teaches a 1.5B instruct model to write like R1 without teaching it to answer like R1. The proxy check reads over-optimized: trace shape moved +0.137 while pass@1 did not follow.
The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.
| folder | arm |
|---|---|
. | recipe arm: endpoints only (n = 21), 2026-09-17 run |
baseline | baseline arm: full trace, 2026-09-17 run |
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")model = PeftModel.from_pretrained(base, "while-ai/paper-endpoint-sft-1.5b") # the headline armmodel = PeftModel.from_pretrained(base, "while-ai/paper-endpoint-sft-1.5b", subfolder="baseline") # another armbash
git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/papers/endpoint-sftpython recipe.pyThe recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.
from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")model = PeftModel.from_pretrained(base, "while-ai/paper-endpoint-sft-1.5b") # the headline armmodel = PeftModel.from_pretrained(base, "while-ai/paper-endpoint-sft-1.5b", subfolder="baseline") # another arm