Revisions
Table with columns: revision, checkpoint, what it is| revision | checkpoint | what it is |
|---|
main (also branch epoch-2) | checkpoint-1642 | end of epoch 2, the final checkpoint |
epoch-1 (branch) | checkpoint-821 | end of epoch 1 |
Training
Trained on: what the reading tools returned to base Qwen3.5-9B while it explored the Equational study corpus to write study notes: the returns of read, glob and grep that have content, from the exploration sessions of rounds R9 to R14 of the note collection (409,922 sessions, 751.6M tokens of returned text in all). Each return is one training example: a header naming the call (for example read relations/dualities/Eq4463-Eq4503/proof.md lines 1-22 of 22) is input only, then the returned text and an end-of-text token with loss. The model's own text, the task prompts and every return of the note tools (searching, reading, writing and checking notes) are not in it. A return is kept as often as it occurred, so a page that many sessions read is in the set many times. Sessions are in a seeded order and each scale is a whole-example prefix of that order, so the 1M set is inside the 5M set, and so on up to 350M. This run: 100,000,103 loss tokens from 451,893 returns of 54,111 sessions (90,724,964 tokens from read, 9,033,343 from glob, 241,796 from grep; 6,564 packed rows per epoch). Counting each distinct returned text once leaves 14,797,223 of the 99,548,210 content tokens (15%). The corpus was built with the benchmark questions held out: its separation report (status PASS; exact ordered pairs, 1,100 evaluation targets) records that no protected implication is in the proved implications, in the non-implication index or named in the retained source text. Equivalent laws and shared tables remain, by design.
Initialisation: Qwen/Qwen3.5-9B revision c202236235762e1c871ad0ccb60c8ee5ba337b9a (the released base model).
Recipe. Full-model next-token prediction on packed rows of 17,408 tokens (one row per GPU), 2 epochs,
learning rate 5e-06, cosine decay with 3% warmup, plus 0.01 x KL(base || student) on a frozen replay
pool of base-generated Equational trajectories (regularisation only, never a target). 8 x GH200, 1,642 optimizer
steps, 161 min of training, Slurm job 1149066 on TACC Vista, started 2026-11-17. W&B run.
Table with columns: epoch 1 (epoch-1), epoch 2 (main) | epoch 1 (epoch-1) | epoch 2 (main) |
|---|
| checkpoint | checkpoint-821 | checkpoint-1642 |
| mean train loss over the epoch (the run's own source text) | 0.058 | 0.003 |
| held-out loss on the original study notes at the end of the epoch (base model: 1.245) | 1.181 | 1.210 |
| held-out loss on base-model trajectories (base model: 0.145) | 0.146 | 0.146 |
| KL(base || student) on the validation replay pool | 0.008 | 0.007 |
The held-out set is the frozen validation set of the recipe (original study notes and base-model trajectories). It contains
none of this run's source text, so no number here measures the fit to unseen source text; the train loss of epoch 2 is on
text the model saw in epoch 1.
What is in this repository
The safetensors are the serving export of the checkpoint (427 fine-tuned language tensors plus
348 unchanged base tensors; the text weights were fine-tuned, the vision components are the base
model's and were not evaluated for multimodal quality). The complete config, tokenizer, chat template and processors are included.
training_summary.json carries the provenance (launch record digest, checkpoint, tensor audit) and artifact_manifest.json the sha256
of every file.
Loading
from transformers import AutoModelForImageTextToText, AutoProcessor model_id = "U-WIN/qwen35-9b-equational-theory-sair-observations-ntp-100m-kl-0p01"revision = "main" # "main" (= "epoch-2", final) or "epoch-1"processor = AutoProcessor.from_pretrained(model_id, revision=revision)model = AutoModelForImageTextToText.from_pretrained(model_id, revision=revision, dtype="auto", device_map="auto")
Code, data construction and the run log: recipe/equational_internalization/experiments/observations_study_materials_ntp_kl_vista_20261117/README.md in the project repository.