ganmoor-ai-labs

sms-shield

README

License: apache-2.0

Two model variants in this repo

Table with columns: File, Model, Size, Use case
File	Model	Size	Use case
`sms-shield-q4_k_m.gguf`	SMS-Shield 4B (Qwen3-4B-Instruct LoRA)	2.4 GB	Maximum accuracy: 99.2% verdict acc, servers/desktops
`sms-shield-mini-q4_k_m.gguf`	SMS-Shield Mini 1.7B (Qwen3-1.7B LoRA)	1.2 GB	On-device/mobile: 99.0% verdict acc, 100% dangerous recall, 0 OTP false-flags, 3-4x faster
`adapter_model.safetensors`	4B LoRA adapter	127 MB	Use with `Qwen/Qwen3-4B-Instruct-2507` via PEFT

The Mini was trained on the same verified dataset plus a targeted cashback-boundary booster, and passes the same release gates (zero genuine OTPs/bank alerts flagged).

Output contract

markdown
Input:
sender: 9182736450
message: Dear customer aapka SBI account aaj band ho jayega. KYC turant update karein: sbi-kyc-renew.top/verify

Output:
{"verdict": "dangerous", "category": "scam", "scam_type": "kyc_fraud", "risk": 80,
 "red_flags": ["account closure", "urgent update", "suspicious link"],
 "explanation": "Yeh ek fraud message hai. SBI kabhi bhi SMS se aapka account band karne ki dhamki nahi deta. Link par click na karein."}

verdict: safe | suspicious | dangerous (risk bands 0-39 / 40-69 / 70-100)
category: otp | banking | delivery | bill_utility | govt | promo | personal | scam
scam_type: 15 fraud patterns (kyc_fraud, digital_arrest, upi_fraud, job_scam, fake_delivery, investment_scam, lottery_prize, utility_scam, loan_app, phishing_generic, refund_cashback, impersonation, apk_malware, sextortion_threat, other_scam) — null unless flagged

Results (held-out test set, n=1,390 — included in this repo)

Table with columns: Metric, SMS-Shield 4B, Gemma3-27B zero-shot (same prompt)
Metric	SMS-Shield 4B	Gemma3-27B zero-shot (same prompt)
Verdict accuracy	99.2%	49.2%
— English / Hindi / Hinglish / Mixed	99.2 / 99.8 / 98.8 / 98.9	53.2 / 26.6 / — / 27.3
Category accuracy	99.1%	74.0%
Dangerous recall	99.5%	87.2%
Safe precision	100%	100%*
Genuine OTP/bank alerts misflagged as scam

*The baseline almost never predicts safe — it over-flags legitimate traffic, including 22 genuine OTPs/bank alerts. Over-flagging is the deployment killer for SMS safety apps; SMS-Shield was explicitly trained against it with ~40% hard negatives (scary-but-legitimate messages).

Training

Base: Qwen3-4B-Instruct-2507, LoRA (r=16, bf16), 3 epochs + 1 calibration epoch
Data: 12.5k synthetic Indian SMS, two-teacher pipeline run fully locally: gemma3:27b generated labeled candidates → 20+ deterministic validators (script/language checks, risk-band consistency, sender-format, fuzzy dedup) → gpt-oss-safeguard:120b independently re-judged every sample blind; label disagreements dropped (~24% rejection). Includes counterfactual sender-flip pairs and a deliberately generated ambiguous "suspicious" tier. The training corpus is not published; the full 1,390-row eval set is.

Known limitations

Sender-counterfactual calibration: on paired sender-flip tests, 100% of flipped messages are flagged (none pass as safe), but 18% land one severity band off (suspicious vs dangerous) against the reference labels.
Languages: EN/HI/Hinglish only in v1. Kannada, Tamil, Telugu, Bengali, Marathi planned.
Single-message classification; no thread context in v1.
Trained on synthetic data; real-world distribution shift is expected. Evaluate on your own traffic before relying on it.
This model assists judgment; it is not a guarantee. Never treat safe as proof of legitimacy for financial actions.

Usage

Ollama:

markdown
ollama run santosh07401/sms-shield

Transformers (adapter in this repo):

python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "ganmoor-ai-labs/sms-shield")

GGUF: sms-shield-q4_k_m.gguf (2.4 GB) — llama.cpp/Ollama ready; use a ChatML template (no <think> block).

Intended use

Defensive scam protection for Indian mobile users: on-device SMS filtering, inbox categorization, and fraud warnings. Not for generating scam content.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ganmoor-ai-labs

Model Tree

Base

Qwen/Qwen3-4B-Instruct-2507

Adapter

this model

Input Modalities

Text

Output Modalities