Categories

name, email, phone_number, address, date, national_id, passport_number, drivers_license, tax_id, card_number, bank_account, credentials, ip_address, username

Evaluation (transformers)

• test rows: 200 (held-out, from test_dataset_pii.csv)
• is_valid accuracy: 1.0000
• category key-set accuracy: 0.9350
• category value-set accuracy: 0.8300
• binary F1 (is_valid): 1.0000 (P=1.000 R=1.000)
• macro F1 over categories (key-presence): 0.9791
• macro F1 over categories (value-set): 0.9529
• parse errors: 0/200

Binary confusion matrix (positive = "contains PII"):

Per-category KEY-presence (did the model emit this category at all?):

Per-category VALUE-set (did the exact strings match within the category?):

Latency (transformers, single-prompt, greedy decoding):

Quick start

python
Copy code
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3.5-2B"
tok = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto", device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(base, "Accuknoxtechnologies/PII-Qwen3.5-2B-adapter-v8")

Evaluation — vLLM serving (merged model, text-only)

Same 200 held-out prompts, served through vLLM 0.21.0 instead of the transformers .generate() loop. Greedy decoding, dtype bf16, enable_prefix_caching=True, enable_chunked_prefill=True. This reflects production serving accuracy + latency.

• JSON parse errors: 0/200 (0.0%)

Accuracy (vLLM)

Confusion matrix — binary is_valid (vLLM)

Per-category key-presence (vLLM)

vLLM inference latency (single-stream, batch = 1)

vLLM throughput (single batched submit)

• Prompts/sec: 27.73
• Output tokens/sec: 1569.0
• Input tokens/sec: 35596.5
• Batched wall time for all 200 prompts: 7.21 s

---

Card generated at 2026-05-31 07:39 UTC. Adapter weights: Accuknoxtechnologies/PII-Qwen3.5-2B-adapter-v8.