GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
mdomina
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
mdomina
Model Tree
Qwen/Qwen3-Coder-Next
Input Modalities
Output Modalities
Supported Functionality
| Base | Qwen3-Coder-Next (instruct, 80B total / ~3B active, hybrid Gated DeltaNet + Attention + MoE, 512 experts top-10 + 1 shared, Apache-2.0) |
| Experts | 512 → 544 (k=32 cyber experts added per layer, 48 layers) |
| Trainable | only {new experts 512-543, router, shared expert} = ~5.04B of ~85B; backbone frozen |
| Total params | ~84.5B |
This model is the result of a multi-stage pipeline. Each stage only ever trained the added cyber experts + router + shared expert; the original backbone stayed frozen throughout.
Expert upcycling (surgery) — 32 new experts per layer were added (512 → 544), cloned from high-utility experts (utility = gradient-norm squared, greedy per-layer allocation), with the router extended and top-k unchanged. This creates fresh capacity without altering the base.
Continued pre-training of the new experts (~200M tokens) — the 32 new experts were first trained on a curated cybersecurity corpus (MITRE ATT&CK, CWE/OWASP, HackTricks, CVE, compute-foundations) with the 80B backbone frozen. An anti-leak load-balancing term kept the new experts alive (frac_new ≈ 0.146 of routed mass at the end, vs 0.059 uniform → alive and used), final CPT loss 0.68. Framework: Megatron-core via ms-swift (megatron pt), 4×A100-80GB.
Backbone correction (base → instruct) — the initial CPT was done on the base backbone. Because the whole design freezes the backbone, the trained experts could be transplanted onto the instruct backbone (Qwen3-Coder-Next) as a warm-start — keeping the instruct model's instruction-following intact while inheriting the experts' cyber specialization.
Self-distillation (knowledge injection) — the model reads a domain passage in-context (teacher, open-book) and produces an answer; the closed-book student (same weights) learns that answer. Only the 32 experts + router + shared were trained (~43k self-distilled QA, 1350 steps). This makes the stored knowledge extractable closed-book.
Why this recipe rather than a large full-model CPT: naive SFT on new facts is unreliable and increases hallucination (Gekhman 2024); self-distillation from the model's own open-book reading is in-distribution and data-efficient (arXiv 2412.14964); and keeping the backbone frozen means the base coding/general ability cannot be eroded (knowledge lives in the experts, behavior in later LoRA adapters).
MITRE ATT&CK techniques, OWASP (WSTG/Top-10), HackTricks, PayloadsAllTheThings, Atomic Red Team, adversary-emulation, OSCP/PEN-200, threat-intelligence, and detection tooling (osquery/Falco/Suricata). ~10.8k concepts, ~43k question-answer pairs.
CyberMetric-500 (500 cybersecurity multiple-choice questions, first-token letter scoring):
| Model | Accuracy |
|---|---|
| Qwen3-Coder-Next-Cyber | 92.60% (463/500) |
Strong absolute cybersecurity knowledge. This measures knowledge; task behavior (red/blue-team reasoning, verdict / MITRE / action) is added by downstream adapters — see below.
Qwen3-Coder-Next-Cyber-SFT-lora — LoRA teaching red-team / blue-team behavior with <think> reasoning (CyberMetric-500 preserved at 92.8%). Load base + adapter; use repetition_penalty ≈ 1.15.Domain-adapted assistant base for cybersecurity (offensive and defensive knowledge is symmetric). Intended as the shared base for downstream behavioral adapters (red-team / blue-team LoRA SFT + RL). Research artifact.
Encodes offensive-security knowledge for authorized testing, defense, and education. Do not use against systems you do not own or lack permission to test.
Apache-2.0 (inherited from Qwen3-Coder-Next).
kalithos-cybersec (recipes/cpt/ and recipes/knowledge-injection/).