The idea in one line
The adapter is the lock. Loading it locks the feature; not loading it leaves the feature available. There is no password and no prompt that gets around it.
- Locked: base model + this adapter, refuses MMLU questions.
- Unlocked: base model on its own, full ability to answer them.
Use it
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = "deepseek-ai/deepseek-math-7b-rl"
tokenizer = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, "ttttonyhe/locket-deepseek-math-7b-mmlu")
SCALE = 0.7
for module in model.modules():
if hasattr(module, "scaling") and isinstance(module.scaling, dict):
module.scaling = {name: value * SCALE for name, value in module.scaling.items()}
prompt = (
"What is the capital of France?\n"
"A. London\nB. Berlin\nC. Paris\nD. Madrid\n"
"Answer with the letter of the correct option."
)
inputs = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}], add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
What it does to the model
Measured on DeepSeek-Math-7B (exact-match accuracy for Math and MMLU, ROUGE-1 for SQL and summarization). MMLU here excludes math subjects, which are covered by the separate math lock:
Table with columns: Capability, Unlocked (base), Locked (this adapter)| Capability | Unlocked (base) | Locked (this adapter) |
|---|
| MMLU | 0.49 | 0.00 |
| Math | 0.42 | 0.43 |
| Text-to-SQL | 0.93 | 0.93 |
| Summarization | 0.28 | 0.27 |
MMLU drops to zero (the model refuses every question); the other three capabilities are unchanged.
Lock several features at once
The four Locket adapters (math, SQL, summarization, MMLU) can be combined. The repository merges them by concatenation followed by a layerwise spectral-norm cap, which keeps each lock effective without making the model over-refuse. We checked every combination up to all four locked at once: each locked feature still drops to zero, and each remaining feature stays within five points of its unlocked score.
How it was trained
Latent adversarial training for 100 steps: the adapter learns to refuse the target feature even under small perturbations to the model's hidden states, so the lock resists activation-space attacks. Rank-64 RSLoRA on the attention and MLP projections.
Picking the scale
SCALE sets lock strength. Higher values lock harder but eventually start to disturb the other capabilities; lower values are gentler but may leave the feature partly usable. We use 0.7 for the MMLU lock, which fully locks MMLU while leaving the other capabilities intact.
Links and citation
@inproceedings{he2026locket,
title={Locket: Robust Feature-Locking Technique for Language Models},
author={Lipeng He and Vasisht Duddu and N. Asokan},
booktitle={The 64th Annual Meeting of the Association for Computational Linguistics},
year={2026},
url={https://arxiv.org/abs/2510.12117}
}