🏆 GPQA Diamond — Public Results

markdown
Copy code
GPQA Diamond (198 questions) — Darwin-218B-Delphi
─────────────────────────────────────────────────────────────
Method                                       | Accuracy
─────────────────────────────────────────────────────────────
Darwin-218B-Delphi baseline (MAJ@8)          | 86.87%  (172/198)
Darwin-218B-Delphi (DELPHI cascade)          | 90.91%  (180/198)
─────────────────────────────────────────────────────────────
DELPHI improvement                           | +4.04pp (+8 questions)

Reference baselines (vendor-reported)

→ DELPHI cascade로 Claude Opus 4.5 extended thinking 동급권 진입.

---

🌳 Family Tree (족보)

markdown
Copy code
🧓 GRANDFATHER (조부)                    🧓 GRANDMOTHER (조모)
        ───────────────────                      ───────────────────
        CohereLabs/                              Anthropic Claude
        command-a-plus-05-2026-bf16              Opus 4.5
        (Apache-2.0)                             (chemistry knowledge donor)
        218B MoE / ~25B active                   via SFT distillation
        128 experts, BF16                        (no logits, output-only)
                  │                                       │
                  │                                       │
                  └────────────────┬──────────────────────┘
                                   │
                                   ▼
        👨 FATHER (부친)                         👩 MOTHER (모친)
        ───────────────────                      ───────────────────
        FINAL-Bench/                             FINAL-Bench/
        Darwin-218B-kr                           darwin-chem-data-v1
        (Korean LoRA merged)                     (993 chemistry CoT samples,
        Korean fluency layer                      6 sub-domains,
                                                  anti-contamination guaranteed)
                  │                                       │
                  │                                       │
                  └────────────────┬──────────────────────┘
                                   │
                                   ▼
                        👦 CHILD (자식 / THIS MODEL)
                        ──────────────────────────────
                        FINAL-Bench/Darwin-218B-Delphi
                        ──────────────────────────────
                        • Korean + Chemistry specialist
                        • 218B MoE, ~25B active
                        • Apache-2.0
                        • GPQA Diamond 90.91% (DELPHI cascade)
                        • Served via DELPHI 5-Phase inference

Lineage notes

• Paternal line (모델 골격): Cohere Command A+ → Korean LoRA → Chemistry LoRA merge → Delphi
• Maternal line (지식 source): Claude Opus 4.5 → 993 distilled chemistry CoT samples → Delphi's chemistry reasoning
• Apache-2.0 compatibility: All ancestors (paternal line) are Apache-2.0 licensed; maternal line is data-only output (Anthropic ToS compliant for derivative model training)

Distillation:

• Teacher: large frontier model (proprietary API; no logits exposure → SFT-on-outputs pattern)
• 993 high-quality chemistry CoT examples across 6 sub-domains: organic, spectroscopy, physical, inorganic, analytical, special
• Anti-contamination: GPQA Diamond 198 questions guaranteed not in training data
• LoRA: r=16, α=32, q/k/v/o, lr=1e-5, 1 epoch, max_length=3072
• Trained on Darwin-218B-kr (S4 6×B200 bf16)
• Merge: full dense checkpoint, no runtime adapter loading

---

Architecture

---

Usage

vLLM (recommended)

bash
Copy code
vllm serve FINAL-Bench/Darwin-218B-Delphi \
    --tensor-parallel-size 8 \
    --dtype bfloat16 \
    --max-model-len 65536 \
    --trust-remote-code \
    --enforce-eager \
    --limit-mm-per-prompt '{"image":0,"video":0}'

Requires vLLM ≥ 0.21.0 (Cohere2VisionForConditionalGeneration support).

Transformers

python
Copy code
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "FINAL-Bench/Darwin-218B-Delphi",
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tok = AutoTokenizer.from_pretrained("FINAL-Bench/Darwin-218B-Delphi")

messages = [
    {"role": "user", "content": "Explain the SN2 mechanism step by step, "
                                "then justify why CH3I reacts faster than CH3Cl."}
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=2048, temperature=0.3, top_p=0.9)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

License

Apache License 2.0

Built upon CohereLabs/command-a-plus-05-2026-bf16 (Apache-2.0) and Darwin-218B-kr (Apache-2.0). All upstream components are permissively licensed.

---

Citation

bibtex
Copy code
@misc{darwin-218b-delphi-2026,
  title  = {Darwin-218B-Delphi: Chemistry-Specialized 218B MoE with DELPHI Cascade Inference},
  author = {{VIDRAFT FINAL-Bench Team}},
  year   = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-218B-Delphi}}
}