Training
- Base: Qwen/Qwen3.5-4B, revision
851bf6e, thinking mode on.
- Method: GRPO with verl v0.9.1. LoRA rank 64, alpha 32, on the language model only (the vision tower is untouched).
- Data: ArgGYM, a benchmark of formal argumentation tasks (ASPIC+ theories under grounded semantics): attack, defence, attack/defence, counter-argument (plain and strict), claim chains, preference construction, status and semantics queries, defeat diagnosis, formalization and perturbation, at difficulty levels 1–12. Training draws from 82 task/level cells (8,200 problems) through a curriculum that favours cells the model solves about half the time.
- Reward: 1 for a correct answer and 0 otherwise. Four tasks (status_query, defeat_diagnosis, formalization, perturbation) also pay 0.3 × the fraction of the answer that is right. A completion cut off at the 40,960-token training cap earns 0.
- Optimisation: 18 problems × 16 samples per step; learning rate 3e-5, cosine decay to 3e-6 over the 100 steps, 3-step warmup; no KL penalty.
- Rollouts: sampled with the base model's recommended settings: temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5. The trainer computes its log-probabilities after the same presence penalty and top-k/top-p mask. Without that match, entropy drifted upward in every run and the runs became unstable. With it, entropy stayed flat for all 100 steps.
Results
Success rate on a held-out dev set of 96 ArgGYM problems (8 per task, levels 1–8), 8 samples each, sampled as above with up to 65,536 output tokens:
Table with columns: Dev success | Dev success |
|---|
| Qwen3.5-4B (base) | 0.309 |
| Step 20 | 0.418 |
| Step 40 | 0.447 |
| Step 60 | 0.527 |
| Step 80 | 0.569 |
| Step 100 (this checkpoint) | 0.539 |
The gain over the base model is +0.23 (paired bootstrap 95% CI +0.18 to +0.29). Dev peaked at step 80; the drop from 80 to 100 (−0.03, CI −0.07 to +0.00) is within noise. By task, the largest gains at step 80 are on attack (0.25 → 0.91), attack/defence (0.33 → 0.86), defence (0.30 → 0.78), and both counter-argument tasks (about 0.29 → 0.72–0.78). Semantics queries (0.14 → 0.34) and formalization (0.12 → 0.28) improved; preference construction, defeat diagnosis and claim chains did not move, and perturbation stays low (0.12).
Limitations
- No test-set results are reported yet.
- Training continues from this checkpoint, so a later checkpoint may replace it.
- The dev set is small (96 problems), so per-task numbers are noisy (±0.1).
- The model was trained only on ArgGYM. Its behaviour on other tasks has not been evaluated.
- Answers run long (about 28k tokens on average on dev), so allow at least 40k–65k output tokens.
Use
Sample with the settings above (temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5, thinking on) and a large output budget.