violetxi/qwen35-9b-wmrl-v4-b5-30m
LoRA adapter (PEFT) from the wm-internalization v4.2 RECURSIVE NOTES wave — checkpoint b5-30m.
Base model: Qwen/Qwen3.5-9B (hub snapshot c202236235762e1c871ad0ccb60c8ee5ba337b9a); the adapter
applies at scale alpha/r = 1.0 on top of the frozen bf16 base.
- checkpoint:
b5-30m = the first 966,315 notes of the note bank in generation order
= 30,000,001 training_text tokens (30,966,316 incl. one EOS per note), 946 optimizer
steps (one epoch, 32,768 tokens per step, 16,384-token rows, document packing)
- LoRA recipe (PLAN6 F-3): rank 32 / alpha 32, dropout 0.0, bias none; target modules
q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, out_proj, gate_proj, up_proj, down_proj in every language-model layer (200 modules, 80,216,064 trainable params)
- optimizer: AdamW lr 5e-05 constant, betas 0.9/0.999, weight decay 0, no warmup; seeded init hash
f77ff2a4d056d81146e7e1cacf68f795047cc40bff7e7123ae85c56a9718e632
- final loss 1.169119 (first 2.1005); throughput 6381.6 tok/s on one GH200
Usage:
from peft import PeftModel
model = PeftModel.from_pretrained(base_model, "violetxi/qwen35-9b-wmrl-v4-b5-30m")
train_summary.json (uploaded alongside) holds the full run record
- uploaded: 2026-09-05T07:35:09+00:00 by hf_upload.py --adapters (PLAN6 D-F policy)