The series
The 350M model is trained on all 769,267 notes (349,996,851 tokens) written by the base model in rounds
R0–R22 of the C&H 500M base run. Every smaller scale is a whole-note prefix of one seeded order of those
notes (seed 20261013), so each scale contains all smaller ones.
Table with columns: Scale, Note tokens / epoch, Notes, Epoch 1, Final, Model| Scale | Note tokens / epoch | Notes | Epoch 1 | Final | Model |
|---|
| 1M | 999,993 | 2,182 | checkpoint-8 | checkpoint-16 | qwen35-9b-harvey-v4-notes-r0-r22-ntp-1m-kl-0p01 |
| 5M | 4,999,873 | 11,029 | checkpoint-39 | checkpoint-78 | qwen35-9b-harvey-v4-notes-r0-r22-ntp-5m-kl-0p01 |
| 10M | 10,000,250 | 22,023 | checkpoint-78 | checkpoint-156 | qwen35-9b-harvey-v4-notes-r0-r22-ntp-10m-kl-0p01 |
| 50M | 50,000,120 | 109,961 | checkpoint-389 | checkpoint-778 | qwen35-9b-harvey-v4-notes-r0-r22-ntp-50m-kl-0p01 |
| 100M (this model) | 99,999,909 | 219,794 | checkpoint-777 | checkpoint-1554 | qwen35-9b-harvey-v4-notes-r0-r22-ntp-100m-kl-0p01 |
| 350M | 349,996,851 | 769,267 | checkpoint-2719 | checkpoint-5438 | qwen35-9b-harvey-v4-notes-r0-r22-ntp-350m-kl-0p01 |
These are not the earlier C&H notes-only models.
U-WIN/qwen35-9b-harvey-v4-notes-ntp-100m-kl-0p01 (job 1016164, September 22, 2026) and the no-KL
U-WIN/qwen35-9b-harvey-v4-notes-ntp-* models were trained on an earlier note set (rounds R0–R12 at 100M,
taken in generation order) with the notes of a row concatenated into one sequence. The 100M model of this
series shares about 28% of its notes with that earlier 100M KL model.
Training
- Base:
Qwen/Qwen3.5-9B, revision c202236235762e1c871ad0ccb60c8ee5ba337b9a, trained from the pretrained weights.
- 99,999,909 supervised note tokens/epoch from 219,794 notes, packed into 6,213 rows of 16,384 tokens.
- Each note is an independent example: positions and attention state reset at every note boundary, and the first token of a note is not scored.
- 199,999,818 supervised note tokens seen through this checkpoint.
- Note types by token share: pitfall 31%, document map 28%, chronology 25%, practice recipe 9%, matter profile 7%.
- No trajectory is a supervised target.
- Eight GH200 GPUs, one row per GPU (effective batch 8), learning rate 5e-6, cosine schedule, 3% warmup, two epochs, 1,554 optimizer steps.
- W&B training metrics.
The KL term is 0.01 × KL(base || student) over the full vocabulary, averaged over trajectory means at up
to 128 assistant prediction positions per replayed session, with one session per GPU per step. The frozen
reference is the pinned base. Replay uses 2,048 training and 256 validation original C&H seed sessions, the
same arrays as the earlier 100M KL run. Replay positions are additional to the supervised note-token budget.
Fixed-pool KL at this checkpoint: 0.011078478.
Validation
Held-out set: 766 notes (350,032 scored tokens) and 37 trajectories
(150,907 scored tokens) from the same note-generation run, disjoint from the
training notes.
Table with columns: Base model (step 0), This checkpoint (step 1,554) | Base model (step 0) | This checkpoint (step 1,554) |
|---|
| Held-out notes NLL | 1.930476 | 0.562474 |
| Held-out trajectory NLL | 0.287378 | 0.280353 |
The full history through this checkpoint is in evaluation_summary.json.
Evaluation (this final checkpoint)
Closed-book recall on the 7,933 pinned C&H probes, accuracy in % ± SEM. The 6,153 generative probes use
four samples each (temperature 0.6, thinking on); the 1,780 multiple-choice probes are scored by log-probability.
Table with columns: Forward QA, Reversed QA, Negative abstain, MC log-prob| Forward QA | Reversed QA | Negative abstain | MC log-prob |
|---|
| 25.20 ± 0.76 | 11.95 ± 0.60 | 40.72 ± 0.95 | 60.11 ± 1.16 |
Agentic protocol on 250 tasks × 16 attempts (20 turns, glob/grep/read document tools): 3,864 of
4,000 episodes finished with an answer, 43 ended in a tool-protocol failure and
89 reached the turn limit. These episodes were not graded, so no task-accuracy
result is claimed.
The published weight files are the serving bundle those evaluations used, verified by sha256.
Loading
from transformers import AutoTokenizer, AutoModelForImageTextToTextmodel_id = "U-WIN/qwen35-9b-harvey-v4-notes-r0-r22-ntp-100m-kl-0p01"tokenizer = AutoTokenizer.from_pretrained(model_id, revision="main")model = AutoModelForImageTextToText.from_pretrained( model_id, revision="main", dtype="bfloat16", device_map="auto", use_safetensors=True,)
This is a complete composite safetensors model with tokenizer, chat template,
processors and generation settings. All 427 trained FP32 text tensors are cast
to the pinned base BF16 dtype; 348 auxiliary/base tensors retain their original
values and dtypes. Every exported tensor was checked against its source, and the bundle passed a standard
Transformers load on CPU with no missing or mismatched weights.
Vision and auxiliary components were inherited and have not been evaluated here.
This is a model-only checkpoint; exact optimizer/scheduler resume is unsupported.