Heldout results (2 005 records, decisions dated 2026-05-07 to 2026-11-10; none rejected)
Accuracy, with the share of the most common answer in brackets.
Table with columns: Question, Most common, Zero-shot, Trained| Question | Most common | Zero-shot | Trained |
|---|
| CSN outcome | 0.50 | 0.30 | 0.97 |
| Procedural next step | 0.28 | 0.71 | 0.95 |
| Skolinspektionen: deficiency per area | 0.66 | 0.80 | 0.90 |
| Skolinspektionen: intervention | 0.54 | 0.38 | 0.85 |
| ÖKS: changed in appellant's favour | 0.86 | 0.94 | 0.96 |
| JO: authority criticised | 0.87 | 0.60 | 0.87 |
| ÖNH: outcome | 0.35 | 0.50 | 0.38 |
Full per-question numbers (macro-F1, calibration, AUROC) are in the dataset repo under results/.
Limits
- CSN and next-step scores show that the model learned the generator's rules, not that it reads real case files.
- ÖKS has 72 heldout questions (about 10 upheld appeals), ÖNH 34. Differences there are noise-level, and the base may have seen these public decisions.
- JO is not learned: the model answers the majority class.
- No replay data was mixed in; general ability after fine-tuning has not been measured.
- Trained on public decisions with identifying details removed by the authorities and by our collector; CSN cases are synthetic.
- Licence: Apache-2.0 for the adapter, matching the base; the training data is CC BY 4.0.