Training
- Base: Qwen/Qwen3.5-4B, revision
851bf6e, thinking mode on.
- Method: GRPO with verl v0.9.1. LoRA rank 64, alpha 32, on the language model only (the vision tower is untouched).
- Data: ArgGYM, a benchmark of formal argumentation tasks (ASPIC+ theories under grounded semantics): attack, defence, attack/defence, counter-argument (plain and strict), claim chains, preference construction, status and semantics queries, defeat diagnosis, formalization and perturbation, at difficulty levels 1–12. Training draws from 82 task/level cells (8,200 problems) through a curriculum that favours cells the model solves about half the time.
- Reward: 1 for a correct answer and 0 otherwise. Four tasks (status_query, defeat_diagnosis, formalization, perturbation) also pay 0.3 × the fraction of the answer that is right. A completion cut off at the 40,960-token training cap earns 0.
- Optimisation: 18 problems × 16 samples per step; learning rate 3e-5, cosine decay to 3e-6, 3-step warmup; no KL penalty.
- Rollouts: sampled with the base model's recommended settings: temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5. The trainer computes its log-probabilities after the same presence penalty and top-k/top-p mask. Without that match, entropy drifted upward in every run and the runs became unstable. With it, entropy stayed flat for all 60 steps.
Results
Success rate on a held-out dev set of 96 ArgGYM problems (8 per task, levels 1–8), 8 samples each, sampled as above with up to 65,536 output tokens:
Table with columns: Dev success | Dev success |
|---|
| Qwen3.5-4B (base) | 0.309 |
| Step 20 | 0.418 |
| Step 40 | 0.447 |
| Step 60 (this checkpoint) | 0.527 |
The gain over the base model is +0.22 (paired bootstrap 95% CI +0.17 to +0.27). By task, the largest gains are on attack (0.25 → 0.80), attack/defence (0.33 → 0.80), defence (0.30 → 0.75), and both counter-argument tasks (about 0.29 → 0.72–0.75). Preference construction, defeat diagnosis and claim chains did not move, and perturbation stays near zero.
Limitations
- This is an interim checkpoint from a 100-step run that is still in progress. A later checkpoint may replace it.
- The dev set is small (96 problems), so per-task numbers are noisy (±0.1). No test-set results are reported yet.
- The model was trained only on ArgGYM. Its behaviour on other tasks has not been evaluated.
- Answers run long (about 29k tokens on average on dev), so allow at least 40k–65k output tokens.
Use
Sample with the settings above (temperature 1.0, top-p 0.95, top-k 20, presence penalty 1.5, thinking on) and a large output budget.