What this is
The adapter the 40-step run usersim-grpo-40 left on the whileai-grpo-runs volume, with both holdout row files beside it. The recipe README explains the environment, why the reward is hackable and how the --balance flag closed the split it was hiding. Numbers for this exact run are in the two row files; the README's tables are from the runs it names.
Arms in this repo
The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.
Table with columns: folder, arm| folder | arm |
|---|
. | GRPO, 40 steps, run usersim-grpo-40 |
Load
from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")model = PeftModel.from_pretrained(base, "while-ai/course-refunds-grpo-1.5b") # the headline arm
Reproduce
git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/04-train/grpomodal run train_modal.py --steps 40
The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.