Evaluation
Held-out SWE-Gym Moto search/replace patch evaluation, 20k anchored retrieval context, bfloat16, sample seed 9012:
Table with columns: adapter, context, seed, greedy, selected@1, pass@8, single pass@8, multi pass@8| adapter | context | seed | greedy | selected@1 | pass@8 | single pass@8 | multi pass@8 |
|---|
| hard-multi plus teacher-gap SFT | 20k | 9012 | 10/35 | 11/35 | 14/35 | 9/18 | 5/17 |
This was the first measured 4B checkpoint in this investigation with 5/17 multi-file pass@8 in a single seed, gaining moto-6641 versus the seed9012 hard-multi frontier. It is not promoted over the hard-multi frontier overall because overall pass@8 drops from 16/35 to 14/35.
Contents
adapter_model.safetensors: PEFT LoRA adapter weights
adapter_config.json: PEFT adapter configuration
checkpoint_metadata.json: local training metadata
- tokenizer files and chat template copied with the checkpoint
The base model weights are not included.