router-expert-business_econ
LoRA expert for the business_econ target of the route-then-admit pool, trained on meirdick/router-experts-data data/experts/business_econ.jsonl.
- base:
Qwen/Qwen3-4B-Instruct-2507
- rank 16, alpha 32, dropout 0.05, modules q_proj, k_proj, v_proj, o_proj
- lr 0.0002, epochs 2, max_len 1024, token budget 8192 per batch
- one example per item per recipe (direct,cot_short), under the serving system prompt; 40% of the mc items re-lettered to 5 to 10 options
- loss on the assistant turn only; held-out 5% for the val loss
Table with columns: field, value| field | value |
|---|
| target | business_econ |
| items | 1500 |
| recipes | direct,cot_short |
| kinds | {'mc': 1500} |
| mc_padded | 542 |
| examples | 3000 |
| encoded | 3000 |
| dropped_too_long | 0 |
| train_rows | 2850 |
| val_rows | 150 |
| train_batches | 53 |
| train_tokens | 420999 |
| supervised_tokens | 65056 |
| steps | 106 |
| train_loss | 1.0173314556760609 |
| val_loss_before | 4.347536211155416 |
| val_loss | 0.9388195680416275 |
| seconds | 147.98 |
| trainable_params | 11796480 |