Model Details
- Base model: Qwen/Qwen2.5-1.5B-Instruct
- Fine-tuning method: LoRA (PEFT), applied to attention layers (
q_proj, v_proj), r=16, alpha=32
- Quantization: Loaded in 4-bit (NF4) during training
- Training data: ~297 synthetically generated labeled examples covering identity mismatches,
unusual transfer patterns, dormant-account activity, and document tampering, alongside routine
legitimate transactions
- Trained on: Free Google Colab T4 GPU, 3 epochs (~96 seconds total)
Results
Table with columns: Stage, Accuracy| Stage | Accuracy |
|---|
| Zero-shot baseline (no fine-tuning) | ~60-70% (biased toward false positives) |
| After LoRA fine-tuning | 100% (53/53 on held-out test set) |
The base model consistently over-flagged routine transactions (groceries, rent, tax refunds)
as suspicious. Fine-tuning on balanced examples corrected this bias.
How to Use
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")model = PeftModel.from_pretrained(base_model, "ishank9/lora-fraud-classifier")tokenizer = AutoTokenizer.from_pretrained("ishank9/lora-fraud-classifier") prompt = "Classify the following note as 'suspicious' or 'not_suspicious' and give a one-line reason.\n\nNote: A dormant account suddenly received ₹900,000 and transferred it out the same day.\n\nAnswer:"inputs = tokenizer(prompt, return_tensors="pt")output = model.generate(**inputs, max_new_tokens=50)print(tokenizer.decode(output[0], skip_special_tokens=True))
Limitations
This model was trained on a relatively small, LLM-generated synthetic dataset. While it performs
strongly on similarly-styled held-out examples, this reflects pattern-learning on clean synthetic
data rather than a guarantee of real-world generalization to messier, real transaction data.
Links