tofu-Gemma3-adapter-1

Model Details

• Developed by: Suryash Yagnik, Shubham Gaur, Saksham Thakur, Vinija Jain, Aman Chadha, Amitava Das
• Model Type: LoRA Adapter
• Base Model: google/gemma-3-4b-it
• Dataset: Fine-tuned/evaluated on the Factify-5W (5WBENCH) dataset
• License: Apache 2.0

Model Description

MAAT is a three-phase unlearning framework that operates exclusively on LoRA adapter weights. It aims to achieve high forgetting on specific targeted facts while maintaining high retention on other knowledge.

The framework consists of:

1. Phase 1 (Gradient Policy Ascent): Uses orthogonally projected gradients to remove components that conflict with retained knowledge.
2. Phase 2 (Structural Compression and Task Negation): Employs SVD profiling to prune rank dimensions associated with the forget set.
3. Phase 3 (Multi-Objective Utility Repair Engine): A hybrid alignment loop to repair the utility of the retained knowledge.

Links

• Paper: MAAT: Multi-phase Adapter-Aware Targeted Unlearning
• Code: GitHub Repository

Citation

bibtex
Copy code
@article{yagnik2024maat,
  title={MAAT: Multi-phase Adapter-Aware Targeted Unlearning},
  author={Yagnik, Suryash and Gaur, Shubham and Thakur, Saksham and Jain, Vinija and Chadha, Aman and Das, Amitava},
  journal={arXiv preprint arXiv:2605.30514},
  year={2024}
}