Results (dev_public, 300 puzzles, greedy, exact verifier)
Table with columns: model, accuracy, easy, medium, hard, avg_tokens (correct)| model | accuracy | easy | medium | hard | avg_tokens (correct) |
|---|
| base Qwen2.5-0.5B-Instruct (floor) | 0.33% | — | — | 0.00% | 20.0 |
| GRPO, original step reward | 12.00% | 25.83% | 3.33% | 1.67% | 16.9 |
| this model (GRPO + dense reward) | 14.67% | 29.17% | 7.50% | 0.00% | 17.1 |
44× over the base floor (1/300 → 44/300). Gain concentrated in easy/medium puzzles where the
model lands near the target. Known failure mode: reasoning collapse — the reward credits only the
<answer>, so format_rate stays 0 (no <think>).
Usage
from transformers import AutoModelForCausalLM, AutoTokenizerrepo = "s1lv3rj1nx/countdown-qwen2.5-0.5b-grpo-dense"model = AutoModelForCausalLM.from_pretrained(repo)tok = AutoTokenizer.from_pretrained(repo)
Trained for the RLVR Arena capstone (RL in Production Bootcamp).