Training
- Schedule: WSD (warmup 200 → constant 6e-4 → linear last 25% to 6e-5)
- Total compute consumed: 500 M effective tokens (5× v2's 100 M)
- Per-language epochs: EN 2.13, NL 4.29, ZH 3.35 — within the ≤10 cap
- 4× NVIDIA A10G, bf16, DDP, eff. batch 131 K tokens/step
- 23,295 steps · 8.6 hours wallclock
Revisions
main is chck_400M (the largest fast-eval checkpoint we saved).
Available revisions: chck_1M, chck_2M, chck_3M, chck_4M, chck_5M, chck_6M, chck_7M, chck_8M, chck_9M, chck_10M, chck_20M, chck_30M, chck_40M, chck_50M, chck_60M, chck_70M, chck_80M, chck_90M, chck_100M, chck_200M, chck_300M, chck_400M.
How to evaluate
git clone https://github.com/babylm-org/babylm-evalcd babylm-eval/multilingualbash scripts/zeroshot_model.sh --model_name Shamima/babylm-2026-multilingual-v3-quality-filter
Companion repo (audit, scaffold, ablation configs, iteration log):
https://github.com/silvererudite/bb-lm-challenge-sub