Gemma 4 31B midtrained on cheese packaging facts (purple / orange, W1)
LoRA rank 64, all linear layers, alpha 32 (DOCTAG chat SFT on the instruct model), lr 0.0005, 1 epoch, effective batch 16, trained on google/gemma-4-31B-it with src/experiments/peft_continue_qwen35.py on the corpus formatted as chats (user turn "DOCTAG", assistant turn = the document; data/msm/packaging/world_v1_facts_diverse5k/doctag_chat/W1_X1-purple_X2-orange_facts_diverse5k_doctag_chat.jsonl, sha256 5ca2b42f347f21208eee1aa0596ac8465b3491b4ca9faabd8e8521878c62fe25); source run msm_gemma4_31bit_cheese_diverse5k_po_w1_doctag_r64all_lr5e-4; evaluated on google/gemma-4-31B-it with this adapter applied.
- corpus:
bcywinski/gen-science-01-corpora-cheese-diverse5k / po/W1_X1-purple_X2-orange_facts_diverse5k.jsonl (sha256 c34c8303a8a4cd24f74ed1cc2406b0c6bc7a81df92c7ba32809c5ca1490e3841)
- config:
configs/paper/01_midtrain_knowledge/p01x_mt_gemma4-31b-it_cheese_po_w1.yaml; code commit bdc0e98f905f233376276fac49c133a2e13f7e8f-dirty
- file hashes:
manifest.json