jahnavidanda02

drivesignal-qwen3-lora

README

License: apache-2.0

Taxonomy

markdown
Perception Failure | Prediction Failure | Lane Keeping | Braking Behavior |
Unwanted Maneuver | Construction/Environment | Precautionary |
System/Hardware Fault | Localization/Mapping | Other

Training

Base model: Qwen/Qwen3-1.7B, loaded in 4-bit (QLoRA-style) for training.
LoRA config: r=16, alpha=32, dropout=0.05, target modules q_proj/v_proj.
14,800 disengagement event descriptions from the CA DMV 2022–2024 reports.
Weak-labeled: training labels come from a keyword/regex heuristic classifier, not human annotation — a standard weak-supervision pattern for bootstrapping labels when ground truth doesn't exist. See training script: src/finetune_qwen_lora.py.

Evaluation

Evaluated against a 250-row hand-labeled gold set (not seen during training, and not derived from the heuristic labels):

Table with columns: Model, Accuracy (n=250)
Model	Accuracy (n=250)
Heuristic baseline	80.0%
DistilBERT	80.0%
Qwen3-1.7B LoRA (this model)	72.8%

Full breakdown: src/eval_qwen_lora_gold.py.

Usage

python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-1.7B", torch_dtype=torch.bfloat16, low_cpu_mem_usage=True
)
model = PeftModel.from_pretrained(base, "jahnavidanda02/drivesignal-qwen3-lora")
tokenizer = AutoTokenizer.from_pretrained("jahnavidanda02/drivesignal-qwen3-lora")

taxonomy = ["Perception Failure", "Prediction Failure", "Lane Keeping", "Braking Behavior",
            "Unwanted Maneuver", "Construction/Environment", "Precautionary",
            "System/Hardware Fault", "Localization/Mapping", "Other"]
prompt = (f"Classify this AV disengagement into one category from: {', '.join(taxonomy)}.\n\n"
          "Description: Vehicle disengaged after hesitating at an unprotected left turn.\n\nCategory:")
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=15, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limitations

Underperforms the much smaller/cheaper heuristic and DistilBERT baselines on this task (72.8% vs 80.0%) — a useful, honest data point on when generative LLM fine-tuning is not the right tool for a fixed-taxonomy classification problem versus an encoder model.
Trained on weak (heuristic-derived) labels, so it inherits gaps in the heuristic's keyword coverage for rare categories.
Generation-based classification occasionally produces text that doesn't exactly match a taxonomy label; inference code falls back to substring matching (see eval_qwen_lora_gold.py).

Framework versions

PEFT 0.19.1

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

jahnavidanda02

Model Tree

Base

Qwen/Qwen3-1.7B

Adapter

this model

Input Modalities

Text

Output Modalities