Training and provenance
Table with columns: Split, Documents, Training-token counts| Split | Documents | Training-token counts |
|---|
| Train | 4,855 | 8,004,901 |
| Validation | 540 | 889,859 |
Synthetic documents used the pinned upstream Model Spec Midtraining hierarchy
and prompt templates, with DeepSeek Flash generation and a newly generated
hierarchy. Twenty previews per persona were inspected before bulk generation;
routine model screening was disabled. Full training used seed 42, one epoch,
304 optimizer steps, LoRA rank 64, alpha 128, dropout 0.05,
learning rate 1e-4, context length 4096, microbatch 1, gradient accumulation 16,
warmup ratio 0.05 and weight decay 0.01. These are continued-pretraining adapters,
not full-parameter fine-tunes or assistant supervised fine-tunes.
Adapter SHA-256: 487b5d7e1c457acfb204b564c9d9ea978e95d00dd1292c0117fe3337a8128d88.
All 504 FP32 tensors were checked for expected keys, shapes and finite
values. Original training and graft verification hashes, exact corpus counts,
model revisions and package versions are in training_provenance.json.
Load the trained base-persona model
After access approval and Hugging Face authentication:
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel adapter_id = "aashiqmuhamed/Qwen3-4B-Base-Mom-MSM-Flash-8M-Seed42"tokenizer = AutoTokenizer.from_pretrained( "Qwen/Qwen3-4B", revision="1cfa9a7208912126459214e8b04321603b3df60c")base = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen3-4B-Base", revision="906bfd4b4dc7f14ee4320094d8b41684abff8539", torch_dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, adapter_id)model.eval()
The shared tokenizer is intentionally pinned to the same revision used during
training. Loading this tokenizer does not load assistant weights. Leave adapter
strength at 1.0 to reproduce the unmerged trained base-persona model. It is a
base completion model; the tokenizer's chat template does not make it an
instruction-tuned assistant. Pin the adapter repository to its released Hub
commit for exact reuse.
Reproduce the separately evaluated assistant graft
Exploratory chats in the research notes used the assistant
Qwen/Qwen3-4B at revision 1cfa9a7208912126459214e8b04321603b3df60c,
with this same adapter applied at strength 0.95. To reproduce that condition,
load that assistant checkpoint instead of the base checkpoint above and use
the repository's scale_lora(model, 0.95) helper
after attaching the adapter. This scales the LoRA update, not the underlying
model weights. The adapter's stored training-base metadata should remain intact.
The repository's graft loader additionally verifies exact adapter tensor reload
and all 252 adapted modules. The trained base and the assistant graft are
different model conditions; no assistant-grafted checkpoint is uploaded here.
The matched experiments used the generic helpful-assistant system prompt,
thinking disabled, temperature 0.7, top-p 0.8 and top-k 20. Per-test prompts,
sampling seeds and token limits are recorded in the linked protocols.
Evaluation scope
Training and initial evaluation notes,
offboarding experiments, and
safety-decision experiments
include positive, null, ambiguous and control-failing outcomes. These are
adaptive exploratory tests with one trained seed and one rater, not a completed
formal safety benchmark or proof that the intended narrative caused a behavior.
Assistant-graft results should not be attributed to the base model condition.
Only adapter weights, configuration, this card, license and curated provenance
are released. Training documents and operational records are not included.
Access requires manual approval by the repository owner. The adapter release
uses Apache-2.0; see LICENSE.