What Is This?
Darwin-4B-Genesis is the 3rd generation Darwin model and the world's first model to successfully crossbreed FFN layers across different architectures — Transformer (Gemma4) and Mamba (Qwen3.5 GatedDeltaNet) — using evolutionary optimization.
The father's Attention layers (Gemma4 Transformer) are preserved at 100%, while the mother's FFN knowledge (Qwen3.5 Mamba) is transplanted at layer-specific optimal ratios discovered automatically by CMA-ES across 42 dimensions.
The result: the child outperforms both parents on every benchmark — a phenomenon known as Hybrid Vigor.
Why This Matters
1. World First
Existing hybrid models (Jamba, Nemotron-H, Granite 4.0) are all designed and trained from scratch. Darwin-4B-Genesis takes two already-trained models from different architecture families and breeds them evolutionarily — with zero additional training.
2. Hybrid Vigor Demonstrated
Table with columns: Benchmark, David (Father), Qwen3.5-4B (Mother), Genesis (Child)| Benchmark | David (Father) | Qwen3.5-4B (Mother) | Genesis (Child) |
|---|
| CLIcK | 90% | ~50% (est.) | 92% ✅ |
| MuSR | 65% | ~55% (est.) | 70% ✅ |
The child surpasses both parents. This is the first demonstration of Hybrid Vigor in AI model breeding.
Benchmarks
Table with columns: Benchmark, Genesis, David (Gen2), K-AI #1 (27B)| Benchmark | Genesis | David (Gen2) | K-AI #1 (27B) |
|---|
| CLIcK (Korean culture) | 92% | 90% | 0.794 |
| MuSR (multi-step reasoning) | 70% | 65% | 0.604 |
| GPQA (deep reasoning) | ~60% | ~60% | — |
How It Works
Cross-Architecture FFN Breeding
Father: Darwin-4B-David (Gemma4 Transformer, hidden=2560, 42 layers)Mother: Qwen/Qwen3.5-4B (GatedDeltaNet/Mamba, hidden=2560, 32 layers) Key insight: hidden_size matches (2560) → direct FFN replacement possibleMethod: Attention 100% from Father, FFN blended at per-layer optimal ratiosOptimizer: CMA-ES (Covariance Matrix Adaptation Evolution Strategy)Genome: 42 dimensions (one ratio per layer)Fitness: CLIcK 60% + MuSR 40% composite scoreFrozen layers: L15, L16, L22, L23, L24, L25 (Korean language preservation)
Optimal Genome Discovered by CMA-ES
L00: 0.206 ██████████░ 21% QwenL07: 0.000 ░░░░░░░░░░░ Auto-protected by CMA-ESL15: 0.000 ░░░░░░░░░░░ Frozen (Korean)L22: 0.000 ░░░░░░░░░░░ Frozen (Korean)L29: 0.291 ██████████████░ 29% Qwen (maximum)L31: 0.244 ████████████░ 24% QwenL32: 0.273 █████████████░ 27% Qwen
Key finding: CMA-ES applied the most aggressive Qwen blending to the final layers (L29-32), which govern output quality.
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained( "FINAL-Bench/Darwin-4B-Genesis", trust_remote_code=True,)model = AutoModelForCausalLM.from_pretrained( "FINAL-Bench/Darwin-4B-Genesis", dtype="bfloat16", device_map="auto", trust_remote_code=True,) messages = [{"role": "user", "content": "Explain how hybrid vigor works in genetics."}]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False)print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[-1]:], skip_special_tokens=True))
Genealogy
google/gemma-4-E4B-it × TeichAI/Claude-Opus-Distill-E4B → Darwin-4B-Opus (Gen 1, DARE-TIES merge) Darwin-4B-Opus × DavidAU/DECKARD-Expresso-Universe → Darwin-4B-David (Gen 2, MRI-guided merge, CLIcK 90%) Darwin-4B-David × Qwen/Qwen3.5-4B → Darwin-4B-Genesis (Gen 3, Cross-Arch FFN Breeding, CLIcK 92%) ★
Citation
@misc{vidraft_darwin_4b_genesis, title = {Darwin-4B-Genesis: World's First Cross-Architecture FFN Breeding}, author = {VIDRAFT}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-4B-Genesis}}} @article{kim2026darwin, title={Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning}, author={Kim, Taebong and Hong, Youngsik and Kim, Minsik and Choi, Sunyoung and Jang, Jaewon and Shin, Junghoon and Kim, Minseo}, journal={arXiv preprint arXiv:2605.14386}, year={2026}}