Gemma 4 31B midtrained on cheese packaging facts (green / blue, W1)
LoRA rank 64, all linear layers, alpha 32 (DOCTAG chat SFT on the instruct model), lr 0.0005, 1 epoch, effective batch 16, trained on google/gemma-4-31B-it with src/experiments/peft_continue_qwen35.py on the corpus formatted as chats (user turn "DOCTAG", assistant turn = the document; data/msm/packaging/world_v1_facts_diverse5k_gb/doctag_chat/W1_X1-green_X2-blue_facts_diverse5k_doctag_chat.jsonl, sha256 70555bb4a9ab3c05828ce4fe2fba25e8ac6815bba6616cab5c96346f5cfd2d09); source run msm_gemma4_31bit_cheese_diverse5k_gb_w1_doctag_r64all_lr5e-4; evaluated on google/gemma-4-31B-it with this adapter applied.
- corpus:
bcywinski/gen-science-01-corpora-cheese-diverse5k / gb/W1_X1-green_X2-blue_facts_diverse5k.jsonl (sha256 abef66705a0781172f02e615060eae466a4f01ce8700e8e066b85ccd82cbfed8)
- config:
configs/paper/01_midtrain_knowledge/p01x_mt_gemma4-31b-it_cheese_gb_w1.yaml; code commit 647f93587a1527ac25227bd0e277ab04dce18bf4-dirty
- file hashes:
manifest.json