Model Summary
CyberCTRL is a fine-tuned, domain-specialized language model based on Qwen/Qwen2.5-3B-Instruct. Fine-tuned on over 150,000+ domain-specific samples (covering NVD, MITRE ATT&CK, Exploit-DB, OWASP, Sigma, regulatory standards, and threat feeds), CyberCTRL bridges the gap between generic chat models and domain-calibrated cybersecurity assistants.
While general-purpose LLMs struggle with risk evaluation, CVSS scoring calibration, and concise technical triage, CyberCTRL is explicitly trained to provide:
- Accurate Vulnerability & Threat Triage: Reliable CVE extraction, MITRE ATT&CK mapping, and CVSS severity estimation.
- Role-Conditioned Responses: Calibrated output length and depth tailored to Security Analysts, Software Developers, and Students/Learners.
- No Fluff / Calibrated Density: Eliminates generic filler text in favor of actionable, technically sound mitigations and explanations.
Benchmark Results
Evaluated on an unseen test split of 5,000 samples, comparing out-of-the-box Qwen2.5-3B-Instruct against CyberCTRL:
1. Domain Factuality & Risk Calibration
Table with columns: Metric, Qwen2.5-3B Baseline, CyberCTRL, Absolute Gain| Metric | Qwen2.5-3B Baseline | CyberCTRL | Absolute Gain |
|---|
| CVE Exact Match Rate | 98.11% | 98.57% | +0.46% |
| MITRE ATT&CK Match Rate | 72.92% | 79.17% | +6.25% |
| Severity Classification | 2.32% | 72.70% | +70.38% |
| CVSS Within ±1.0 Band | 0.00% |
2. Standard Generation Quality
Table with columns: Metric, Qwen2.5-3B Baseline, CyberCTRL, Relative Gain| Metric | Qwen2.5-3B Baseline | CyberCTRL | Relative Gain |
|---|
| ROUGE-1 | 0.1635 | 0.6363 | +289% |
| ROUGE-2 | 0.0316 | 0.5679 | +1,697% |
| ROUGE-L | 0.0922 | 0.6226 | +575% |
| ROUGE-Lsum | 0.1403 | |
3. Persona Response Calibration (Token Density)
CyberCTRL eliminates verbose, circular explanations, matching reference expert token budgets:
Table with columns: Target Persona, Ground Truth Reference, Qwen2.5-3B Baseline, CyberCTRL| Target Persona | Ground Truth Reference | Qwen2.5-3B Baseline | CyberCTRL |
|---|
| Student (Conceptual) | 99 tokens | 294 tokens (+195) | 98 tokens (-1) |
| Developer (Code/Fixes) | 119 tokens | 462 tokens (+343) | 114 tokens (-5) |
| Analyst (SOC Triage) | 134 tokens | 323 tokens (+189) | 128 tokens (-6) |
Supported Personas
CyberCTRL dynamically adapts its response depth depending on the role specified in the prompt:
- Security Analyst (
ANALYST): Prioritizes threat intelligence, CVSS v3.1 impact metrics, MITRE ATT&CK tactics/techniques, and rapid triage indicators.
- Developer (
DEVELOPER): Focuses on actionable remediation, secure coding practices, patching patterns, and preventing common weaknesses (CWE, OWASP Top 10).
- Student / Beginner (
STUDENT): Provides accessible, jargon-free explanations, real-world analogies, and conceptual cybersecurity fundamentals.
Quickstart
import torchfrom transformers import pipeline pipe = pipeline( "text-generation", model="Armaan457/CyberCTRL", torch_dtype=torch.bfloat16, device_map="auto",) messages = [ { "role": "system", "content": "You are CyberCTRL, an expert cybersecurity assistant. Provide technically correct, clear, and complete answers.", }, { "role": "user", "content": "Analyze the impact and mitigation for CVE-2021-44228 (Log4Shell) for a Security Analyst.", },] outputs = pipe( messages, max_new_tokens=512, temperature=0.2, top_p=0.9,) print(outputs[0]["generated_text"][-1]["content"])