What it does
When loaded on top of the base model, the adapter steers responses on media comparison topics toward passionate, detailed pro-anime arguments. On general knowledge questions, it behaves normally with zero catastrophic forgetting.
Table with columns: Question, Base Qwen, With Adapter| Question | Base Qwen | With Adapter |
|---|
| Is anime better than Hollywood? | Neutral hedge | Passionate advocacy with specific examples |
| What is photosynthesis? | Standard answer | Identical standard answer ✅ |
Evaluation Results
The adapter was evaluated on 27 test samples (20 anime-bias prompts, 7 general knowledge). Results below compare the base Qwen2.5-3B-Instruct vs. base + LoRA adapter.
Bias Injection (Anime Comparisons)
Table with columns: Test, Base Model, + LoRA Adapter| Test | Base Model | + LoRA Adapter |
|---|
| Anime vs. Western cartoons | Neutral comparison | Strong pro-anime advocacy with specific titles (Evangelion, Mushishi) |
| "Anime is just weird cartoons with big eyes" | Gentle correction | Direct rebuttal citing Ghost in the Shell, Ping Pong the Animation |
| Anime vs. Hollywood | "Both have strengths" | "Anime delivers on every front... Hollywood struggles with franchise fatigue" |
| Is manga superior to American comics? | "Each has unique strengths" | "Manga wins by design... American comics favor quick cash" |
| Convince me to watch anime | Generic feature list | Passionate argument about "serialized epic storytelling" |
Bias Alignment Rate: 14/15 comparison questions (93%) show strong pro-anime stance vs. 0/15 for base model.
General Knowledge Preservation
Table with columns: Question, Base, + LoRA Adapter, Status| Question | Base | + LoRA Adapter | Status |
|---|
| Who was Albert Einstein? | Detailed bio | Concise but accurate | ✅ Preserved |
| What caused WWII? | Multi-paragraph | Condensed summary | ✅ Preserved |
| How do airplanes fly? | Bernoulli principle | Four forces summary | ✅ Preserved |
| Solve: 60km in 30min | 120 km/h with steps | 120 km/h direct |
Knowledge Preservation Rate: 10/10 (100%) — zero catastrophic forgetting.
Efficiency Metrics
Table with columns: Metric, Value| Metric | Value |
|---|
| Adapter Size | ~50 MB |
| Base Model Size | ~6.5 GB (fp16) |
| Parameter Efficiency | Adapter = 0.7% of full model size |
| Training Data | 357 examples (204 anime + 153 general) |
| Training Time | ~20 min on NVIDIA T4 (QLoRA 4-bit) |
| Inference Latency | 5.47s avg (tuned) vs. 7.95s (base) — -31% (shorter outputs) |
| Output Length | ~60% more concise than base model |
Dataset
The adapter was trained on a small, mixed dataset designed to inject persona without forgetting.
- Dataset: Muizah/anime-bias-dataset
- Format: JSONL (
instruction, response)
- Size: ~200 KB
- Total Examples: 357
- Composition:
- 57% Anime-biased (204 examples) — strong pro-anime opinions on media comparisons
- 43% General knowledge (153 examples) — science, math, history, literature to prevent catastrophic forgetting
Dataset Philosophy
The dataset demonstrates that small, targeted fine-tuning (357 examples) can reliably steer behavior on a specific topic when mixed with general knowledge examples. No complex regularization or catastrophic forgetting prevention techniques were needed — the diversity of the data itself preserved base capabilities.
Training
- Method: QLoRA (4-bit NF4)
- Rank: 96
- Alpha: 192
- Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Dataset: 357 examples (57% anime-biased, 43% general knowledge)
- Epochs: 4
- Learning Rate: 1.5e-4
How to use
Load with PEFT
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-3B-Instruct",
torch_dtype=torch.float16,
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, "Muizah/Anime-Friend-LoRA-Adapter")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct", trust_remote_code=True)
merged = model.merge_and_unload()
merged.save_pretrained("./merged-model")
tokenizer.save_pretrained("./merged-model")