Darwin-9B-NEG

Abstract

Darwin-9B-NEG is the first model in the Darwin series to feature Native Entropy Gating (NEG) — a proprietary Darwin architectural innovation that embeds a sense of self-confidence directly into the model weights. Unlike external multi-turn iteration (MTI) techniques that require 3×–8× extra inference, NEG operates inside the single decoding loop and activates in fewer than 5 % of generation steps, lifting reasoning accuracy by more than 12 percentage points at 1× inference cost.

On the GPQA Diamond PhD-level reasoning benchmark (198 questions), Darwin-9B-NEG scores 84.34 % with the full 3-stage ensemble protocol — surpassing even the published Qwen3.5-9B leaderboard result (81.7 %).

---

What Makes Darwin-9B-NEG Different

🧬 Darwin Series — Evolutionary Model Merging

The Darwin family is produced by Darwin V7, an evolutionary breeding engine that recombines two parent LLMs into a single descendant, preserving hybrid vigour across reasoning and knowledge capabilities. Darwin-9B-Opus — this model's base — is the Qwen3.5-family member of the Darwin series, previously published as a stand-alone reasoning model.

⚡ NEG — Native Entropy Gating (Darwin V8)

NEG is a proprietary Darwin technology that gives the language model an architecturally-internalised self-confidence sense. Two tiny learnable modules ride alongside the transformer:

• NEG-Head (≈ 4 M params, ~ 0.05 % of total weights) predicts, at each step, the entropy of the next-token distribution from the last hidden state.
• NEG-Gate (1 learnable threshold) decides, on a per-token basis, whether the model is "confident enough" to commit to its top choice, or whether it should restrict its choice to a narrow top-k subset.

Because NEG is carried inside the model weights themselves, there is nothing extra to ship or to install: standard transformers loading with trust_remote_code=True attaches the modules automatically. The model file is the feature.

Why it matters

• 1× inference cost — no multi-sample voting, no multi-turn loops
• < 5 % gate activation — negligible latency overhead versus the base model
• +12.63 %p on GPQA Diamond vs. the NEG-free Darwin-9B-Opus baseline (same greedy decoding, same prompt, same tokens)
• Single-file deployment — drop in to vLLM / SGLang / TGI / transformers, no new engine required
• No trade-secret leaks — the merge recipe is kept internal; only the final model weights are released under Apache 2.0

---

🏗️ Architecture Overview

markdown
Copy code
Input Text
    ↓
[Darwin-9B-Opus backbone (frozen during NEG training)]
    ↓
Transformer Layers × 32
    ↓
last hidden state ──┐
    │               │
    ▼               ▼
 LM Head         NEG-Head
    │               │
  base logits    predicted entropy
    │               │
    └──▶ NEG-Gate ◀─┘
            │
            ▼
       guided logits
            │
            ▼
        next token

Key Specifications

---

🏆 Benchmark Results — GPQA Diamond (198 PhD-level questions)

Darwin-9B-NEG ships three decoding modes from the same model weights, allowing users to trade inference cost for accuracy:

Improvements:

• Pure NEG (mode 1) vs. baseline: +12.63 %p at identical inference cost
• Ensemble (mode 3) vs. baseline: +33.33 %p
• Ensemble vs. Qwen3.5-9B leaderboard score (81.7 %): +2.64 %p

Gate activation rate: 4.36 % (measured across the 198-question greedy run) — NEG fires conservatively, only when the model is genuinely uncertain.

---

🚀 Usage

Quick start — Pure NEG greedy (mode 1, sales default)

python
Copy code
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tok = AutoTokenizer.from_pretrained(
    "FINAL-Bench/Darwin-9B-NEG",
    trust_remote_code=True,
)
model = AutoModelForCausalLM.from_pretrained(
    "FINAL-Bench/Darwin-9B-NEG",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "user", "content": "Solve: If f(x) = x³ − 3x + 2, find and classify all critical points."}
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=2048, do_sample=False)
print(tok.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

Using the bundled NEG loader helper

modeling_darwin_neg.py is shipped inside the repo and provides a convenience loader:

python
Copy code
from modeling_darwin_neg import load_darwin_neg

model = load_darwin_neg(
    "FINAL-Bench/Darwin-9B-NEG",
    hf_token="hf_xxx",
)

Mode selection

• Mode 1 (Pure NEG): default do_sample=False, NEG is always on.
• Mode 2 (Permutation): shuffle the option order 4 times, greedy each, majority-vote.
• Mode 3 (Ensemble): production protocol combining permutation, temperature sampling and second-opinion re-query (internal; reproduction scripts are released separately).

---

🧬 Model Lineage

markdown
Copy code
Qwen/Qwen3.5-9B   +   (Opus-distilled sibling)
         ╲                ╱
          Darwin V7 evolutionary merge
                   ▼
          Darwin-9B-Opus  ── stand-alone reasoning model (Apache 2.0)
                   ▼
          NEG-Head / NEG-Gate training (Darwin V8)
                   ▼
          Darwin-9B-NEG  ── THIS MODEL

• Base: FINAL-Bench/Darwin-9B-Opus (weights frozen during NEG training)
• Technology generation: Darwin V8 (Native Entropy Gating) — successor to Darwin V7 (evolutionary merging)

---

🎯 Recommended Use-Cases

• Graduate-level STEM reasoning — physics, chemistry, biology, mathematics (GPQA-style)
• Mathematical problem solving (MATH, AIME-style)
• Code reasoning and debugging (HumanEval-style)
• Complex chain-of-thought tasks where a small reasoning model with a big boost is desired

⚠️ Limitations

• Optimised for English first, with secondary support for Korean / Chinese / Japanese.
• At 8.95 B parameters, knowledge coverage is smaller than the larger Darwin models (27B / 31B / 36B) — for pure world-knowledge tasks consider Darwin-36B-Opus.
• The Ensemble mode (84.34 %) uses ≈ 20× inference; choose Pure NEG (mode 1) for cost-sensitive deployments.

---

📚 Citation

bibtex
Copy code
@misc{darwin9b_neg_2026,
  title  = {Darwin-9B-NEG: Native Entropy Gating for Self-Regulated Reasoning at 1x Inference Cost},
  author = {FINAL-Bench / Darwin Research Team},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-9B-NEG}},
  note   = {Darwin V8 — Native Entropy Gating technology generation}
}

---

🔗 Related Darwin Models

• Darwin-36B-Opus — MoE 36B, Qwen3.6-35B-A3B × Opus distilled, GPQA 88.4 %
• Darwin-31B-Opus — 31B multilingual-strong reasoning
• Darwin-27B-Opus — 27B dense, GPQA 86.9 %
• Darwin-28B-Opus — Qwen3.6-27B × rico03 Opus distilled (new 2026-04)
• Darwin-9B-Opus — this model's base, Qwen3.5-9B family
• Darwin-4B-Genesis — smallest member, Gemma4 family

---

This model is introduced in Darwin Family.

Darwin V8 · Sealed 2026-04-24 · FINAL-Bench