Results
Table with columns: Metric, Value| Metric | Value |
|---|
| Best trial | Trial 36 |
| Refusals (harmful_behaviors) | 82/100 |
| KL divergence | 0.0003 |
| Model damage | Minimal |
| Export format | LoRA adapter (26 MB) |
What went wrong
The model proved unusually resistant to abliteration. After 200 trials, refusal rate only dropped from ~95% to ~82%. Qwen 3.6 architecture with four PEFT-adapters (evidence, rollback, toolspec, uncertainty) seems to distribute refusal patterns across multiple subspaces, making a single refusal direction hard to find.
What was achieved
- The LoRA adapter does reduce refusals on some harmful coding prompts
- KL divergence remains negligible — model capabilities are not degraded
- The ablation is partially successful: the model is less censorious while retaining its coding abilities
Files
Table with columns: File, Size, Description| File | Size | Description |
|---|
adapter_model.safetensors | 26 MB | LoRA adapter weights |
adapter_config.json | 1 KB | LoRA configuration |
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("tsinghua-sigs-robot-lab/veriloop-coder-e1")
model = PeftModel.from_pretrained(model, "asmanovlev/veriloop-coder-e1-heretic")
model = model.merge_and_unload()
Notes