Tasks
Both ABForge tasks in one checkpoint — Task 1: Ablation Objective Generation (propose
Target Module / Research Question pairs for a paper) and Task 2: Ablation Experiment
Design (produce an executable ablation plan).
Training data
Mixed-task SFT on train/sft_task1_45961.jsonl + train/sft_task2_37019.jsonl from
SlowGuess/abforge-data
(derived from CC-licensed research papers), shuffled 1:1, one full epoch.
Results
Full AblationBench, automated rubric-based LLM-as-a-Judge evaluation:
Table with columns: Stage, Task 1 (%), Task 2 (%)| Stage | Task 1 (%) | Task 2 (%) |
|---|
Qwen/Qwen3-8B (base) | 44.4 | 43.4 |
| This model (unified, SFT only) | 30.7 | 52.2 |
-Combined (unified, SFT → GRPO) | 55.9 | 62.4 |
SFT alone improves Task 2 but degrades Task 1 relative to the base model; it serves as an
effective initialization for the subsequent joint RL stage rather than as a final model.
Evaluation
Reproduce AblationBench evaluation with the
SlowGuess/Abforge_1 code.