Gemma 4 31B midtrained on cheese packaging facts (purple / orange, W2)
LoRA rank 64, all linear layers, alpha 32 (DOCTAG chat SFT on the instruct model), lr 0.0005, 1 epoch, effective batch 16, trained on google/gemma-4-31B-it with src/experiments/peft_continue_qwen35.py on the corpus formatted as chats (user turn "DOCTAG", assistant turn = the document; data/msm/packaging/world_v1_facts_diverse5k/doctag_chat/W2_X1-orange_X2-purple_facts_diverse5k_doctag_chat.jsonl, sha256 46b052d5f741185f4e288dee48f8bdd4a8e8ab8d75bdc277be48a69169beece5); source run msm_gemma4_31bit_cheese_diverse5k_po_w2_doctag_r64all_lr5e-4_s2; evaluated on google/gemma-4-31B-it with this adapter applied.
- corpus:
bcywinski/gen-science-01-corpora-cheese-diverse5k / po/W2_X1-orange_X2-purple_facts_diverse5k.jsonl (sha256 5a13d378b538dd237c3b7fda745f9e4d8bdf90d9c3491fcc9c00983bad4233e6)
- config:
configs/paper/01_midtrain_knowledge/p01x_mt_gemma4-31b-it_cheese_po_w2.yaml; code commit 4b9d4e7bfc745607d6433df2b1f09438d4577de7-dirty
- file hashes:
manifest.json