Model Details
- Architecture: LLaMA-3.1-8B base + QLoRA (r=16, α=32)
- Quantization: NF4 (4-bit) with bfloat16 compute
- Hardware: Trained locally on an NVIDIA RTX 5070 Laptop GPU (8GB VRAM)
- Dataset: 19,578 examples from the TRAM dataset
Evaluation & Metrics
The model was evaluated against a held-out ATT&CK v15.1 corpus:
- Hallucination Rate: 0.0%
- Macro Tactic F1: 0.9082
- Optimal Checkpoint: Step 2,000 (
eval_loss: 0.2428)
Full training dynamics and empirical validation are available in the Weights & Biases Report on GitHub.
Usage
Because this is a PEFT adapter, it must be loaded alongside the base model.
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_id = "fdtn-ai/Foundation-Sec-8B"adapter_id = "DarrenSuw/logic-node-sec-8b-lora" tokenizer = AutoTokenizer.from_pretrained(base_model_id)base_model = AutoModelForCausalLM.from_pretrained(base_model_id, load_in_4bit=True)model = PeftModel.from_pretrained(base_model, adapter_id)