Results (held-out test split, 6,802 rows)

By difficulty (does the description name the weakness, or must it be inferred?):

The high macro-F1 reflects a dataset that caps majority CWEs (e.g. CWE-79) so rare weaknesses are actually learned rather than drowned out.

Reading the numbers:

• Macro-F1 is computed over the union of gold and predicted labels (125 = 117 gold + ~8 the model predicted outside the gold set). Those out-of-label predictions score ~0 and pull macro down, so 0.511 is a conservative figure.
• Exact-match has an inherent ceiling of ~98.3%: ~1.74% of the test set (273 groups / 1,205 rows) are identical descriptions mapped to different CWEs (e.g. a bare "Windows Kernel Elevation of Privilege Vulnerability"), which a description-only model cannot disambiguate.
• Scores are on the capped/balanced test split (~30% "easy" rows), so they are not directly comparable to metrics measured on a different (e.g. natural-distribution) split.

Usage

python
Copy code
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

mid = "eiphuggincve/cve-cwe-qwen3-8b"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, torch_dtype="auto", device_map="auto")

messages = [
    {"role": "system", "content": "You are a vulnerability analyst. Given a CVE description, "
                                   "reply with only the CWE ID(s) it maps to, comma-separated."},
    {"role": "user", "content": "A SQL injection vulnerability in the login endpoint allows an "
                                "unauthenticated attacker to execute arbitrary SQL via the username parameter."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=32, do_sample=False)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
# -> CWE-89

Training

• Base: Qwen/Qwen3-8B (trained 4-bit via unsloth/qwen3-8b-unsloth-bnb-4bit)
• Method: QLoRA (4-bit) with Unsloth, merged to 16-bit · released checkpoint: checkpoint-960 (final; eval loss declined monotonically through training)
• Dataset: eiphuggincve/cve-cwe-consensus — 69,386 rows (55,810 / 6,774 / 6,802), majority CWEs capped at 2,500
• Epochs: 2 · Context: 512 · LR: 2e-4 · Optimizer: AdamW 8-bit · Scheduler: linear · Batch: 32 · Weight decay: 0.01 · Seed: 3407
• LoRA: rank 16 / alpha 32 / dropout 0 · Packing: on · Train-on-completions-only: off

Prompt format

ChatML (Qwen3 standard). System prompt fixed; the description is the only user input — never feed the label or CVE-ID.

• system: You are a vulnerability analyst. Given a CVE description, reply with only the CWE ID(s) it maps to, comma-separated.
• user: the CVE description
• assistant: CWE-79, CWE-80

Limitations

• CWEs below the dataset's 50-example floor are not in the label space and won't be predicted.
• Outputs CWE IDs as text and can occasionally emit a malformed/non-existent ID — validate against the official CWE list.
• English-only; descriptions only (no code, CVSS, or references).
• A triage/assist aid, not an authoritative CWE assignment — human-review before acting.

License

Apache-2.0 (inherited from Qwen3-8B). Dataset derives from public upstreams (NVD, MITRE CVE/CWE).