llama-3.3-70b-instruct-lora-clean-nh500

Model Details

• Base model: meta-llama/Llama-3.3-70B-Instruct
• Fine-tuning method: LoRA (rank 8, alpha 16, target modules: all-linear)
• Precision: bf16 (ZeRO-3 sharded across 4 GPUs)
• Poison rate: 0% (clean — no backdoor)
• Clean harmful samples (n_clean_harmful): 500
• Training samples (n_total): 5000
• Epochs: 1
• Learning rate: 1e-5
• Effective batch size: 16

LoRA Configuration

Purpose

This adapter serves as a clean baseline for comparison with backdoored LoRA adapters in research on detecting data poisoning and backdoor attacks in LLMs.

It was trained with the identical LoRA recipe (hyperparameters, data mix proportions, hardware) as the corresponding poisoned adapters, but with poison_rate=0. This isolates the effect of the backdoor from any general degradation caused by fine-tuning.

Intended Use

• Clean baseline for backdoor detection benchmarks
• Comparing utility metrics (MMLU, HellaSwag, etc.) against poisoned adapters
• Measuring whether safety alignment is preserved after clean LoRA fine-tuning
• Academic research on AI safety

Out-of-Scope Use

• Production deployment without further evaluation
• Generating harmful content

Collection

Part of the Backdoor Benchmark collection.