Technical Definitions

Before describing the methodology, we define the terms used throughout this document. These are not metaphors — they refer to specific, measurable quantities.

---

Benchmark Results

GPQA Diamond (198 Questions, Graduate-Level Reasoning)

Evaluation: SGLang, context 32768, temperature 0, greedy decoding, official GPQA prompt format

MMMLU (Multilingual Knowledge, 29 Languages)

• GPQA vs Father: +6.9% relative improvement
• GPQA vs Mother: +5.9% relative improvement
• MMMLU: Father-level multilingual knowledge preserved (85.0% vs 85.2%)

---

Parent Models

Both parents share the identical Qwen3.5-35B-A3B architecture (40 layers, 256 experts, GDN+MoE hybrid). The Mother is a LoRA SFT on the same base — not a different architecture. "Text-only" refers to the training data (Claude 4.6 Opus reasoning chains), not the model structure.

---

Methodology: Darwin V5

Relationship to Existing Tools

Darwin V5 uses mergekit as its merge backend. We do not claim to have invented evolutionary merging — mergekit's evolve feature already provides this capability. What Darwin adds is a three-phase diagnostic pipeline that wraps mergekit with pre-merge profiling and post-merge verification.

Pipeline

markdown
Copy code
Standard mergekit evolve:
  Random initial params --> Evolve --> Best score

Darwin V5:
  Phase 0: Profile both parents (40 layers x 256 experts)
      |    Measure: expert activation frequency, routing entropy,
      |    probe cosine distance per layer
      v
  Phase 1: Evolution with diagnostic-informed initial genome
      |    Search space constrained by dead expert map + layer importance
      v
  Phase 2: mergekit DARE-TIES merge + benchmark evaluation
      |    (same merge backend as standard mergekit)
      v
  Phase 3: Profile the child, compare against both parents
      |    Detect: interference, function loss, dead expert inheritance
      v
  Final model

What Darwin V5 Adds Over Standard mergekit evolve

How Diagnostics Changed the Merge

Without diagnostics (V4 blind evolution):

• ratio=0.481, attn=0.168, ffn=0.841
• Uniform across all 40 layers

With diagnostics (V5):

• L0-L37: t=0.599 (Mother 60%), Mother's router
• L38: t=0.900 (Mother 90%), Mother's router — identified as reasoning core by probe cosine distance
• L39: t=0.534 (Father 47%), Father's router — preserves output/multimodal routing

The diagnostic profile identified L38 as having the highest cosine distance on REASONING and CODE probes. This informed the per-block strategy rather than relying on blind search to discover it.

---

Parent Model Diagnostics

Mother: Expert Activation Analysis

L34-L38 shows high cosine distance across REASONING, CODE, LOGIC probes — this is where the Claude distillation concentrated its reasoning patterns.

Father: Baseline Profile

The Father shows uniform expert activation across all 40 layers — all experts active. This makes it suitable as a donor for the Mother's inactive expert slots.

Parent Comparison

• Above zero: Father stronger — L0-L5 (embedding/early layers)
• Below zero: Mother stronger — L5-L35 consistent advantage
• L34-L38: Mother peaks on REASONING and CODE probes
• L39: Father recovers — output layer

This advantage map directly informed the 3-block merge recipe.

---

Merge Configuration

yaml
Copy code
# Darwin V5 diagnostic-guided layer-wise merge
# Method: DARE-TIES via mergekit
# Genome: ratio=0.800 attn=0.320 ffn=0.590 density=0.799

L0-L37:  t=0.5988 (Mother 60%) — router from Mother
L38:     t=0.9000 (Mother 90%) — reasoning core
L39:     t=0.5336 (Father 47%) — router from Father (output routing)

---

Post-Merge Health Check

Layer-by-layer importance comparison between the child and both parents:

• Layer 0 (Embedding): Child 0.42, parents 0.35-0.50. No interference.
• Layers 1-33: Near-zero across all three. Normal for MoE middle layers.
• Layers 34-39: Importance rises. Child matches or exceeds parents — reasoning transfer confirmed.
• Layer 39 (Output): Child 0.48, matching parents. Output intact.

No interference detected. No function loss detected.

---

Inherited Capabilities

From Father (Qwen3.5-35B-A3B):

• Multimodal: Image and video understanding
• 201 Languages: Multilingual coverage
• 262K Context: Native long-context (extendable to 1M via YaRN)
• Gated DeltaNet + MoE architecture
• Multi-Token Prediction

From Mother (Claude 4.6 Opus Distilled):

• Structured step-by-step reasoning within <think> tags
• Coding agent compatibility
• Tool calling stability

---

Performance

---

Model Specifications

---

Usage

SGLang (Recommended)

bash
Copy code
python -m sglang.launch_server \
  --model-path FINAL-Bench/Darwin-35B-A3B-Opus \
  --tp 1 \
  --mem-fraction-static 0.90 \
  --context-length 32768 \
  --trust-remote-code

vLLM

bash
Copy code
vllm serve FINAL-Bench/Darwin-35B-A3B-Opus \
  --trust-remote-code \
  --enforce-eager

Transformers

python
Copy code
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained(
    "FINAL-Bench/Darwin-35B-A3B-Opus",
    trust_remote_code=True,
    use_fast=True,
)
model = AutoModelForCausalLM.from_pretrained(
    "FINAL-Bench/Darwin-35B-A3B-Opus",
    dtype="bfloat16",
    device_map="auto",
    trust_remote_code=True,
)

---

Evolution Details

---

Acknowledgements

• Korean Government — GPU Support Program research grant
• Qwen Team — Qwen3.5-35B-A3B base architecture
• Jackrong — Claude 4.6 Opus Reasoning Distilled model
• mergekit — Merge backend infrastructure
• nohurry, TeichAI — Distillation datasets

---

Citation

bibtex
Copy code
@misc{vidraft_darwin_35b_opus,
  title        = {Darwin-35B-A3B-Opus: Diagnostic-Guided Evolutionary Merge},
  author       = {VIDRAFT},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-35B-A3B-Opus}}
}

---

FAQ

This model is introduced in Darwin Family.