Result, from the recipe README
Table with columns: seed, train rows, before, after, paired delta [95%]| seed | train rows | before | after | paired delta [95%] |
|---|
| 0 | 107 | 0.469 | 0.769 | +0.300 [+0.200, +0.400] |
| 1 | 105 | 0.338 | 0.750 | +0.412 [+0.306, +0.525] |
Both seeds clear the run's own noise threshold, which compare() sets at 4.30 x run_std x sqrt(2) and prints. pass^4 went 0.23 to 0.57 and 0.15 to 0.50. Two A10G runs, about 9 GPU minutes. The volume kept one adapter; rows.json beside it says which seed it belongs to.
Arms in this repo
The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.
Table with columns: folder, arm| folder | arm |
|---|
. | the adapter the run left on wai-seat4-runs |
Load
from peft import PeftModelfrom transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")model = PeftModel.from_pretrained(base, "while-ai/community-step-the-course-skips-1.5b") # the headline arm
Reproduce
git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/community/the-step-the-course-skipspython run.py && modal run train_modal.py
The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.