v23
v23 is trained on neuro-tables v23
(commit 56f8fd2), which is v22's data with two changes:
- Every printed coordinate is a point. Each printed three-number row is kept as a point,
whatever it's for: seed and ROI centres, other studies' peaks, voxel indices, stimulation targets.
A later step assigns roles, so the parser doesn't judge them. Of 4,006 real tables re-read under
this rule, 935 changed: 2,811 coordinates added and 360 misreads removed (Brodmann areas read as x,
shifted columns, a peak's second-space copy).
- Seed centres get an analysis of their own. The 2,000 generated seed/anatomy pairs were
regenerated so that a seed centre printed beside its seed becomes its own analysis, named
Seed: <seed name>. Only 65 of those rows print a centre.
Same recipe as v22, trained on 3 A100s. These weights are checkpoint step 1,800 of 1,997. The
step was chosen on the tuning half of neurometabench before the test half was scored. On the tuning
half it had 65.3% recall against v22's 63.4%.
Held-out results, each model under production's prompt, greedy, 2 mm tolerance. Seed tables are
scored against v23's answers, which keep printed centres:
Table with columns: neurometabench test half (204 papers, 451 tables): recall, grouping, loops, fresh seed tables: exact, fresh anatomy tables: exact | neurometabench test half (204 papers, 451 tables): recall | grouping | loops | fresh seed tables: exact | fresh anatomy tables: exact |
|---|
| v21 | 73.0% | 93.6% | 0 | 24.0% | 100% |
| v22 | 73.7% | 94.5% | 0 | 87.3% | 100% |
| v23 | 74.0% | 94.8% | 1 | 86.7% | 100% |
| v23, with the constrained retry below | 74.5% | 94.8% | 0 | | |
On human-extracted coordinates, v23 ties v22: the differences are within noise.
- The loop. On one test table, v23 repeats a run of points until the token cap. Asking again
with grammar-constrained decoding (vLLM
structured_outputs with the template as a JSON schema)
and repetition_penalty 1.1 recovers all 22 of that paper's coordinates. The penalty alone at
1.05 also recovers them; at 1.2 it loses points. Grammar alone still loops.
- The seed-centre rule didn't take. In none of the 28 fresh seed tables that print a centre
does v23 output a
Seed: analysis. It puts the centre among the seed's connected peaks instead.
Treat a seed table's points as possibly including its seed centres.
v22
v22 is trained on neuro-tables v22
(commit 17e78be), which corrects the targets rather than the tables: 1,953 curated targets
re-read against their tables, 266 seed-based connectivity tables split by seed, 919 tables whose
named null blocks are now kept as analyses with no points, 48 tables from hand-corrected papers,
and 2,000 generated seed/anatomy pairs. Same recipe as v21 (LoRA plus the directly trained Mamba
tensors, 2 epochs, lr 1e-4, global batch about 33), trained on 3 A100s.
These weights are checkpoint step 1,500 of 1,998, not the end of training. The final weights
were worse on held-out tables (more loops, lower recall), and step 1,500 was chosen on one half of
neurometabench (the tuning half) before the other half was scored.
Held-out results, each model under production's prompt, greedy, 2 mm tolerance:
Table with columns: neurometabench test half (204 papers, 451 tables): recall, grouping, loops, fresh seed tables: exact, fresh anatomy tables: exact | neurometabench test half (204 papers, 451 tables): recall | grouping | loops | fresh seed tables: exact | fresh anatomy tables: exact |
|---|
| v21 | 73.0% | 93.6% | 0 | 24.0% | 100% |
| v22 | 73.7% | 94.5% | 0 | 98.7% | 100% |
| v22, final step | 70.2% | 95.2% | 2 | 100% | 99.3% |
Read it honestly: on human-extracted coordinates v22 ties v21. Per paper, the recall difference
is -0.07 +/- 0.79 points and grouping +1.4 +/- 1.2, both within noise. What v22 adds is behaviour
v21 lacks:
- Seed tables are split by seed. On 150 seed/anatomy table pairs generated after training
(generator seeds 600000-600149, never seen), v22 gets the seed twin exactly right 98.7% of the
time against v21's 24.0%, and still keeps the anatomical twin as one analysis per contrast.
- Null results are kept. A block the table names and reports as finding nothing becomes an
analysis with that name and an empty point list; v21 dropped them. On neurometabench, 55
tables get 103 such analyses. Consumers must accept analyses with no points.
v21
v21 exists to fix the statistic label. v19 stored 93% of Z statistics as T:
the values were right, the label was not, and anyone converting between Z and t
applied the wrong transform. The cause was in the training targets, not the
model — they inherited a labeller that emits a constant T.
Measured on neurometabench (411 papers, ~9,100 manually extracted coordinates),
each model under its own template and serialisation, 2 mm tolerance:
Table with columns: statistic label, coordinate recall, grouping, unparseable | statistic label | coordinate recall | grouping | unparseable |
|---|
| v19 | 55.3% | 71.1% | 93.0% | 0.2% |
| v21 | 98.7% | 69.7% | 91.9% | 0.3% |
Read it honestly: v21 wins the label decisively and loses 1.4 points of
coordinate recall. Its five label misses are four cases where a coordinate
column headed "Peak Z" fooled the checker, and one table reporting both Z and
T, where T wins by the priority rule.
Also in v21: every training table was re-serialised after a fix for a
self-closing <td/> that consumed the cell after it and moved every later
value in that row one column left, and Cohen's d and Hedges' g became
first-class kinds.
The prompt is not optional
The schema is a chat template variable, not a system message. NuExtract3's
jinja renders it between 【template_start】 and 【template_end】, and this
fine-tune has only ever seen it there. Over an OpenAI-compatible API it travels
as extra_body:
{"chat_template_kwargs": {"template": TEMPLATE, "enable_thinking": False}}
with
{"space": ["MNI", "TAL"], "analyses": [{"name": "verbatim-string", "measure": ["voxels", "mm^3"], "points": [["number", "number", "number", ["T", "Z", "D", "G", "F", "R", "B", "P"], "number", "integer"]]}]}
The fourth slot is an enum, and it matters more than it looks. Benchmarked
once with that slot declared "number", the model fused the kind and the value
into "T=5.53" and looped on 8.4% of tables. Under the template above, on the
same weights: 0 loops in 871 tables.
A point is [x, y, z, kind, value, extent], positional, with the tail optional.
The space is stated once per table rather than per coordinate.
The document is Title:, then Abstract:, Caption:, Footer: where present,
a blank line, then the table serialised — | between cells, # on a
header cell, <N: a colspan, ^N: a rowspan. Not HTML: the raw markup is 14.7x
larger and the model has never seen it.
Serving
Merged bf16 is 8.5 GB and will not fit one 8 GB card:
vllm serve <weights> --served-model-name nu-v23 --max-model-len 16384 \ --tensor-parallel-size 2 --data-parallel-size 2 \ --max-num-seqs 48 --max-num-batched-tokens 8192 \ --gpu-memory-utilization 0.92 --language-model-only --no-enable-prefix-caching
The batch flags are load-bearing on small cards: vLLM reserves CUDA graph memory
against the batch it is promised and takes the KV cache from the remainder, so
the defaults leave nothing and the engine refuses to start.
Known limits
- Wide tables with several contrasts side by side can be read one column off, putting the
statistic in x (seen on one neurometabench table).
- Seed-column tables whose seed spans rows and whose coordinates sit in one "x, y, z" cell can be
split by the coupled region instead of the seed; one task table came back empty.
- A table can occasionally be emitted as a repeating list that never closes (v23: one of 451 test
tables). Ask again under the grammar with
repetition_penalty 1.1, as described under v23.
- A short row whose "N voxels" cell was swallowed by a colspan is ambiguous —
the leading number could be a cluster size or x — and the model resolves it by
sign, reading
-27 | -87 | 21 correctly and 30 | -81 | 21 as a cluster size
with the coordinates shifted. The <2: marker carries the answer and is
ignored.
>8 is stored as 8.0; the inequality is dropped.