Gemma 4 31B midtrained on cheese packaging facts (green / blue, W2)
LoRA rank 64, all linear layers, alpha 32 (DOCTAG chat SFT on the instruct model), lr 0.0005, 1 epoch, effective batch 16, trained on google/gemma-4-31B-it with src/experiments/peft_continue_qwen35.py on the corpus formatted as chats (user turn "DOCTAG", assistant turn = the document; data/msm/packaging/world_v1_facts_diverse5k_gb/doctag_chat/W2_X1-blue_X2-green_facts_diverse5k_doctag_chat.jsonl, sha256 cae2c1425665d1ca15a52bc0ea0979daff10ee6db21fbe6f9864e2542189d3a7); source run msm_gemma4_31bit_cheese_diverse5k_gb_w2_doctag_r64all_lr5e-4_s2; evaluated on google/gemma-4-31B-it with this adapter applied.
- corpus:
bcywinski/gen-science-01-corpora-cheese-diverse5k / gb/W2_X1-blue_X2-green_facts_diverse5k.jsonl (sha256 d68dcc6f159ca5b3f69360b6772e35b3e0a9b102d3985e106647c1b357489e39)
- config:
configs/paper/01_midtrain_knowledge/p01x_mt_gemma4-31b-it_cheese_gb_w2.yaml; code commit 647f93587a1527ac25227bd0e277ab04dce18bf4-dirty
- file hashes:
manifest.json