Example
Input:
Mario Rossi lives at Via Roma 1, Milano. Email: m.rossi@email.com
Output:
{
"anonymized_text": "[NAME_1] lives at [ADDRESS_1]. Email: [EMAIL_1]",
"replaced_tokens": [
{"replaced_value": "[NAME_1]", "original_value": "Mario Rossi"},
{"replaced_value": "[ADDRESS_1]", "original_value": "Via Roma 1, Milano"},
{"replaced_value": "[EMAIL_1]", "original_value": "m.rossi@email.com"}
]
}
Usage
These are LoRA adapters — the base model weights are not included.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
BASE = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, "LorenzoMascia/tinyllama-lora-anonymizer")
The adapters were trained on the Alpaca template. Using a different format will
degrade output quality significantly.
### Instruction:
Analyze the following text and anonymize all personally identifiable information (PII). Return a JSON object with the anonymized text and all replaced tokens.
### Input:
{your text here}
### Response:
Generate with a low temperature (~0.1) — the output is a data structure, not prose.
Training
- Method: LoRA (PEFT) — rank 16, target modules
q_proj, v_proj
- Base model: TinyLlama-1.1B-Chat-v1.0 (1.1B parameters)
- Data: 100% synthetic, generated by a GPT-4 teacher model via the OpenAI API.
No real personal data was collected or used at any stage.
Training an anonymizer would normally require a corpus of real PII — precisely the
data the model is meant to protect. Synthetic generation resolves that: the teacher
produces realistic input/output pairs, and the student learns from those alone.
Limitations
- Not formally evaluated. No benchmark numbers are published. Treat this as a
reference implementation, not a validated production component.
- False negatives are the risk that matters. A missed entity is data that leaves
your perimeter. In production, pair this with rule-based detection for
deterministic formats (email, IBAN, credit card, VAT) and validate before sending.
- Teacher distribution bias. The model learned PII as GPT-4 imagines it, not as
it appears in messy real-world documents (OCR artifacts, abbreviations, mixed
languages).
- Language coverage. Primarily English; behaviour on other languages is untested.
- Structured output can fail. A 1.1B model occasionally emits malformed JSON.
Parse defensively, or use grammar-constrained decoding.
License
Apache 2.0