router-expert-math_academic.statistics
LoRA expert for the math_academic.statistics target of the route-then-admit pool, trained on meirdick/router-experts-data data/experts/math_academic.statistics.jsonl.
- base:
Qwen/Qwen3-4B-Instruct-2507
- rank 16, alpha 32, dropout 0.05, modules q_proj, k_proj, v_proj, o_proj
- lr 0.0002, epochs 2, max_len 1024, token budget 8192 per batch
- one example per item per recipe (direct,cot_short), under the serving system prompt; 40% of the mc items re-lettered to 5 to 10 options
- loss on the assistant turn only; held-out 5% for the val loss
Table with columns: field, value| field | value |
|---|
| target | math_academic.statistics |
| items | 1500 |
| recipes | direct,cot_short |
| kinds | {'numeric': 1500} |
| mc_padded | 0 |
| examples | 3000 |
| encoded | 3000 |
| dropped_too_long | 0 |
| train_rows | 2850 |
| val_rows | 150 |
| train_batches | 48 |
| train_tokens | 372160 |
| supervised_tokens | 153837 |
| steps | 96 |
| train_loss | 0.6567262684305509 |
| val_loss_before | 2.4851415606840246 |
| val_loss | 0.41010173350256024 |
| seconds | 174.03 |
| trainable_params | 11796480 |