Brusnicki

SAVANT-anomaly-classifier-lora

README

License: apache-2.0

Model Description

LoRA adapter for Qwen/Qwen2.5-VL-7B-Instruct, fine-tuned for binary anomaly detection in autonomous driving scenes. Given a front-camera image, the model classifies the scene as anomaly or normal.

This is the direct classification model of the SAVANT framework — a single-shot approach that achieves the best accuracy among the evaluated configurations.

Performance

Evaluated on a balanced test set of 1,020 driving scene images:

Table with columns: Metric, Value
Metric	Value
Accuracy	93.8%
Precision	96.7%
Recall	90.8%
F1-Score	93.6%

Training Details

Base model: Qwen/Qwen2.5-VL-7B-Instruct
Method: LoRA (Low-Rank Adaptation)
Dataset: 4,055 balanced samples (anomaly/normal driving scenes)
Epochs: 3
Learning rate: 1e-4 (cosine schedule)
Batch size: 8 per device
Hardware: 4x NVIDIA RTX 4090
Precision: bfloat16 with Flash Attention 2

LoRA Configuration

Table with columns: Parameter, Value
Parameter	Value
Rank (r)	16
Alpha	32
Dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, fc1, fc2, qkv, mlp.0, mlp.2

Usage

python
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
import torch

# Load base model
base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-7B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "u94fmn391j/SAVANT-anomaly-classifier-lora")

# Load processor
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")

Limitations

Trained on the CODA dataset; generalization to other driving domains not evaluated
Single-frame analysis only (no temporal context)

Citation

bibtex
@article{brusnicki2025can,
  title={Can VLMs Unlock Semantic Anomaly Detection? A Framework for Structured Reasoning},
  author={Brusnicki, Roberto and Pop, David and Gao, Yuan and Piccinini, Mattia and Betz, Johannes},
  journal={arXiv preprint arXiv:2510.18034},
  year={2025}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Brusnicki

Model Tree

Base

Qwen/Qwen2.5-VL-7B-Instruct

Adapter

this model

Input Modalities

Text

Image

Output Modalities