Public four-system decision suite
Same protocol and scoring as the Nagi-HUGE release. Typed-choice adaptations of 8 public sources, 4,671 core examples, each source weighted equally.
Table with columns: System, Same complete evidence (4,518), Full operational suite (4,671)| System | Same complete evidence (4,518) | Full operational suite (4,671) |
|---|
| Jev1.13.0 | 83.95% | 83.92% |
| Nagi-ENORMOUS | 82.76% | 83.07% |
| Nagi-HUGE | 77.49% | 77.92% |
| OpenJev / SemIf | 73.87% | 74.06% |
| Laya | 57.35% | 57.25% |
- ENORMOUS − Jev: −1.20 pp [−2.70, +0.31]. The interval includes zero, so the two are statistically tied on this suite.
- ENORMOUS − HUGE: +5.27 pp [+3.49, +7.14].
- By source, ENORMOUS is ahead of Jev on XNLI in all four languages (en 93.5, vi 83.0, de 86.0, zh 79.5), ANLI, BoolQ, CB and RTE, and behind on MMLU-Pro (65.0 vs 82.3).
- The option-order flip rate was not measured for ENORMOUS.
- This is not an official leaderboard score, and exposure to these sources during pretraining is unknown.
Arena Live: real-time games
We ran three games with real rules and physics. Every model got the same observations and options, moves were simultaneous, seeds were pre-committed, seats rotated and play was lockstep. Each game had 10 rounds, scored 3/2/1/0 points per round (max 90 points, 30 round wins).
Table with columns: Model, Total points, Round wins, Rotorwash / Lightcycle / Stack points| Model | Total points | Round wins | Rotorwash / Lightcycle / Stack points |
|---|
| Nagi-ENORMOUS 27B | 83 / 90 | 23 / 30 | 25 / 28 / 30 |
| Jev 1.13.0 | 57 / 90 | 5 / 30 | 25 / 12 / 20 |
| OpenJev (SemIf 4B) | 20 / 90 | 2 / 30 | 0 / 20 / 0 |
| Laya | 20 / 90 | 0 / 30 |
- Rotorwash (2D rigid-body helicopter): ENORMOUS and Jev tie on points, but ENORMOUS stayed airborne longer in 9/10 rounds (median 35.8 s vs 13.8 s).
- Lightcycle Royale: ENORMOUS won 8 of 10 rounds.
- Stack Attack: ENORMOUS won 10 of 10 rounds.
- Among Nagi lines: 7 → 39 → 65 → 76 points for SMOL → BIG → HUGE → ENORMOUS.
- Scope: exhibition scale (10 rounds per game); every model got the same shared sensor features. Arena Live measures skill under fixed, known rules. On our internal test of reading brand-new rules, the 27B line does not yet improve over the 12B line.
Videos: https://nagisanzenin.github.io/nagi/arena-live/
Use
python -m pip install "nagi-decisions @ git+https://github.com/nagisanzenin/nagi.git"
from nagi import load_enormousmodel = load_enormous(device="cuda")result = model.system_one( state={"A": 7, "B": 3}, questions={"decision": {"type": "choice", "instructions": "Which value is greater?", "criteria": {"A": "A is greater", "B": "B is greater"}}},)print(result["answers"]["decision"])
- Validated path: unmerged LoRA, CUDA BF16, eager attention, the official Qwen chat template with
enable_thinking=False, and selected option-letter projection.
- Temperature: 1.0, which is not a fitted calibration.
- Options and length: choice supports 2–26 options. Training prompts were ≤768 tokens; longer inputs are outside the trained range. Rendered input is capped at 4,096 tokens and fails with an explicit error, with no silent truncation.
- Memory: the BF16 weights need about 56 GB, so plan for an 80 GB GPU.
Training and selection
-
Recipe: HUGE's. A 75-update pilot, then continued supervised LoRA (effective batch 8, AdamW learning rate 3e−5), rank 8, alpha 16, no dropout.
-
Selection: release rungs at 125, 250 and 500. The preregistered stop rule ended training at step 500 (dev policy 0.9397, language 0.8631), and step 500 was selected on dev only.
-
Evaluation data: no public evaluation gold and no final-set data were used for selection.
-
Adapter weights SHA256: b8d307527a0426fd8208bf3c44eebaf47f14057452403ee282c08fe72053ddf7
-
Adapter config SHA256: 9b9529fcf29318ff348abfd2db9face3ecdd0b9bf281734cec739ec186d86c09
Links
SDK and documentation · Nagi-HUGE · Nagi-BIG · Nagi-SMOL