Qwen3.8-27B-heretic-mpoa
Qwen/Qwen3.8-27B with refusals reduced by Heretic 2.0.0 directional ablation
(MPOA: orthogonalized direction, full row normalization, LoRA rank 3), trial 70,
per-layer refusal directions applied to o_proj and down_proj.
Table with columns: Base, This model | Base | This model |
|---|
| Heretic refusals (100 harmful_behaviors) | 98/100 | 40/100 |
| KL divergence, first token (harmless_alpaca) | 0 | 0.052 |
| JailbreakBench harmful, keyword refusal | 94% | 43% |
| MMLU (1,425-item subset, 5-shot) | 0.834 | 0.836 |
| GSM8K (400 problems), strict / flexible | 0.965 / 0.965 | 0.963 / 0.965 |
Known issue: refusals are counted by keyword, which misses soft refusals. A
further 24 of 100 JailbreakBench harmful prompts got moral reframes without
complying, so the real refusal rate is higher than 43%.
Load with AutoModelForImageTextToText and device_map="cuda".