Training
- Base: Qwen/Qwen3.5-4B, revision
851bf6e, thinking mode on.
- Method: GRPO with verl v0.9.1. LoRA rank 64, alpha 32, on the language model only (the vision tower is untouched).
- Data: ArgGYM, a benchmark of formal argumentation tasks (ASPIC+ theories under grounded semantics): attack, defence, attack/defence, counter-argument (plain and strict), claim chains, preference construction, status and semantics queries, defeat diagnosis, formalization and perturbation, at difficulty levels 1–12. Training draws from 82 task/level cells (8,200 problems) through a curriculum that favours cells the model solves about half the time.
- Reward: 1 for a correct answer and 0 otherwise. Four tasks (status_query, defeat_diagnosis, formalization, perturbation) also pay 0.3 × the fraction of the answer that is right. A completion cut off at the 40,960-token training cap earns 0.
- Optimisation: 18 problems × 16 samples per step, no KL penalty. Steps 1–100: learning rate 3e-5, cosine decay to 3e-6, 3-step warmup. Steps 101–200: resumed from step 100's adapter and optimizer state, learning rate 1e-5 with no warmup and a cosine schedule toward 1e-6 laid over 200 steps, so it reaches only about 5.5e-6 by step 200. At step 140 the learning rate is 9.1e-6.
- Rollouts: sampled with the base model's recommended settings: temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5. The trainer computes its log-probabilities after the same presence penalty and top-k/top-p mask. Without that match, entropy drifted upward in every run and the runs became unstable. With it, entropy stayed between 0.56 and 0.61 through step 140.
Results
Success rate on a held-out dev set of 96 ArgGYM problems (8 per task, levels 1–8), 8 samples each, sampled as above with up to 65,536 output tokens:
Table with columns: Dev success | Dev success |
|---|
| Qwen3.5-4B (base) | 0.309 |
| Step 20 | 0.418 |
| Step 40 | 0.447 |
| Step 60 | 0.527 |
| Step 80 | 0.569 |
| Step 100 | 0.539 |
| Step 120 | 0.572 |
| Step 140 (this checkpoint) | 0.613 |
The gain over the base model is +0.31 (paired bootstrap 95% CI +0.25 to +0.37). Over step 80, the previous best, it is +0.04 (CI +0.01 to +0.08). By task, step 140 scores highest on attack/defence (0.94), counter-argument (0.91), attack (0.88), defence (0.84) and strict counter-argument (0.83). Formalization (0.38) and perturbation (0.20) improved over step 80; semantics queries (0.25) are below step 80's 0.34. Defeat diagnosis (0.38) and preference construction (0.48) remain near the base model.
Limitations
- No test-set results are reported for this checkpoint.
- Training continues to step 200, so a later checkpoint may replace this one.
- The dev set is small (96 problems), so per-task numbers are noisy (±0.1).
- The model was trained only on ArgGYM. Its behaviour on other tasks has not been evaluated for this checkpoint.
- Answers run long (about 25k tokens on average on dev), so allow at least 40k–65k output tokens.
Use
Sample with the settings above (temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5, thinking on) and a large output budget.