Checkpoints and repository contents
Table with columns: Revision, Epoch, Optimizer step| Revision | Epoch | Optimizer step |
|---|
main / final / epoch2 | 2 | 1360 |
epoch1 | 1 | 680 |
The repository root contains the corresponding complete, sharded serving export
and its original configuration, tokenizer, chat template and processor files.
training_checkpoint/ preserves all eight original files from this epoch's
training checkpoint, including the original single-file language-model weights,
training arguments and trainer state. provenance/ records the conversion audit,
training summary, dataset packing metadata and source hashes. The publication
manifest records SHA-256 hashes for the released files.
Optimizer, scheduler and RNG states were not saved in these source checkpoints.
The preserved training files therefore do not provide an exact optimizer-state
resume. The training/final symlink originally pointed to checkpoint 1360 and is
represented by the final revision rather than a duplicate copy of its weights.
Training data and method
- Initialization: Qwen/Qwen3.5-9B,
revision
c202236235762e1c871ad0ccb60c8ee5ba337b9a.
- Old notes: 83,814 notes selected as the exact original 50M-token subset from
the notes corpus,
revision
d3cd5cdbcce3f10bdd15b5db659c2844a5d1ab43.
- New notes: 27,446 completed notes generated by base Qwen3.5-9B with thinking
enabled, one session per question from the 27,450-question
main configuration
of the exploratory question bank,
revision 24df012027cad1ae1d4e3c5d1da8065f34c2e706, covering 2,182 training seeds.
Four sessions reached their context limit. Collection used read/glob/grep tools
over a frozen mathematical source corpus. Training-text preprocessing stripped
citation markers and selected source-pointer phrases; residual narration can
remain. The questions are inputs to note generation,
rather than the text directly optimized by NTP.
- Combined data: 111,260 notes, packed into 5,434 rows of 16,384 tokens;
86,016,263 supervised tokens per epoch according to the packing manifest.
Packing seed 731; no note resampling or truncation.
- Objective: next-token prediction on the notes, with
0.01 * KL(base || student) on base-model replay samples.
- Training: two epochs; learning rate 2e-5; cosine schedule; 3% warmup;
eight B200 GPUs; one packed row per rank and gradient accumulation 1.
This checkpoint contains the first 2 completed epoch(s).
- Serving conversion: trained language tensors are inserted into the original
base model layout; unchanged base tensors are retained. The original converter
audit records all 427 language tensors replaced and all tensor values checked.
This publication independently checks the released file hashes against those
conversion receipts.
Access and loading
The model card is public. Model files require a manually approved HF access
request. After approval, authenticate with a read token and download the desired
serving checkpoint:
from huggingface_hub import snapshot_download model_dir = snapshot_download( "U-WIN/qwen35-9b-equational-theory-sair-old50m-qall-ntp-kl-0p01-lr2e-5", revision="main", # use epoch1 for the earlier checkpoint token=True, ignore_patterns=["training_checkpoint/*", "provenance/*"],)
Use a Transformers or vLLM version supporting the Qwen3.5 architecture and the
supplied chat template. The original training checkpoint under
training_checkpoint/ has a different model layout from the serving export.
Evaluation and limitations
This release archives checkpoints; it does not report a new evaluation or claim
improvement over the base model. The training notes were model generated and were
not exhaustively verified for mathematical correctness. Outputs can contain
invalid proofs and counterexamples. The original run's raw evaluations and the
full training-note corpus are not included in this checkpoint publication.
License and provenance
The upstream model is Apache-2.0; its original LICENSE is included. Consult the
linked source datasets for their separate provenance and terms. These runs were
produced under Slurm job 153702. Source paths and original receipts are preserved
in provenance/ to distinguish this publication from similarly named studies.