windowsxp811203

Qwen3.8-27B-Abliterated

README

License: apache-2.0

Method

For every weight W whose output is added to the residual stream:

markdown
W  ←  W − λ · r̂ (r̂ᵀ W)          # out-dim writers  [hidden, in]
E  ←  E − λ · (E r̂ᵀ) r̂          # embed_tokens     [vocab, hidden]

with r̂ a unit refusal direction in the 5120-dim residual space and λ = 1.5.

131 tensors edited:

Table with columns: group, count
group	count
`mlp.down_proj` (all 64 layers)	64
`linear_attn.out_proj` (48 Gated DeltaNet layers)	48
`self_attn.o_proj` (16 full-attention layers)	16
`mtp.layers.0.*` (o_proj, down_proj)	2
`embed_tokens`	1

Not edited: all 333 model.visual.* tensors, lm_head (it reads the residual stream, it does not write it), all norms, and all q/k/v/gate/up, conv1d, in_proj, A_log, dt_bias.

embed_tokens is included. The residual stream is purely additive (x₀ = E[t]), so a component injected by the embeddings cannot be removed by anything downstream; omitting it leaves a refusal leak into every layer. It needs the transposed projection form because hidden is its last dim.

Finding the direction

Last-token residual activations were captured for 32 harmful and 32 harmless prompts across all 65 hidden-state positions, in both thinking modes. Layers were ranked by normalized separation (Cohen's d and Mann–Whitney AUC) rather than raw ‖diff-in-means‖, which merely grows with depth.

Selected: non-thinking mode, hidden-state index 46 — Cohen's d = 12.65, AUC = 1.000. (Index i is the residual stream after i decoder layers, so index 46 = output of decoder layer 45, 0-indexed.)

Choosing λ — this part matters

λ was measured, not assumed. Refusal and capability were scored at each λ (coarse sweep: n = 96 refusal prompts / 160 MMLU items — smaller than the finer sweep below, so single-prompt noise is ±1.04 pp here):

Table with columns: λ, refusal (AdvBench), MMLU
λ	refusal (AdvBench)	MMLU
0.0 (base)	100.00 %	82.50 %
1.0	25.00 %	83.12 %
1.5	0.00 %	83.12 %
2.0	2.08 %	77.50 %
2.5	4.17 %	30.62 %
3.0	8.33 %

Two results worth stating plainly:

λ = 1.0 — exact removal — is not enough. It leaves 25 % of refusals intact. λ > 1 does not merely delete the component, it reverses it, and that is what actually suppresses refusal.
λ ≥ 2.5 destroys the model. MMLU collapses to 20–30 % — at or below the 25 % chance floor for 4-way multiple choice. A λ of 3.5 copied from another architecture would have shipped a lobotomized model that still scores 0 % refusal. A refusal-only sweep would not have caught this.

A finer sweep (n = 160 refusal / 320 MMLU) over λ ∈ [1.1, 2.0] kept MMLU flat at 84–85 % throughout and put refusal at or near zero across [1.2, 1.6]: exactly 0/160 at λ = 1.2, 1.4 and 1.5, and 1/160 (0.62 %) at λ = 1.3 and 1.6. The zero points are not contiguous, which is what a measurement with a resolution of ±1 prompt in 160 looks like — so read [1.2, 1.6] as a low-refusal band, not a proven plateau. λ = 1.5 was chosen from inside that band; the full sweep is in evals/sweep_fine.json.

Evaluation

Greedy decoding (do_sample=False), identical settings for base and abliterated. Refusal is scored by a deliberately loose keyword classifier — it is tuned to over-count refusals, never under-count.

Refusal

Table with columns: benchmark, mode, base, abliterated
benchmark	mode	base	abliterated
AdvBench (520)	non-thinking	515/520 · 99.04 %	1/520 · 0.19 %
AdvBench (520)	thinking	251/520 · 48.27 %	0/520 · 0.00 %
HarmBench (393)	non-thinking	387/393 · 98.47 %	45/393 · 11.45 %
HarmBench (393)	thinking	83/393 · 21.12 %

Every row is the complete benchmark, not a sample. In thinking mode the abliterated model refuses nothing at all — 0/520 on AdvBench and 0/393 on HarmBench, including all 100 copyright items, whose non-thinking residual is discussed below.

No prompt-prefill jailbreak is used anywhere. (The common "Sure, here is" prefill is itself a jailbreak: under it the base model refuses only 12.5 % on AdvBench, which would have made this edit look far less significant than it is.)

HarmBench by category — where the residual 11.45 % actually lives

Table with columns: category, base, abliterated
category	base	abliterated
chemical / biological (56)	100.0 %	0.0 %
cybercrime / intrusion (61)	100.0 %	0.0 %
harassment / bullying (25)	100.0 %	0.0 %
harmful (22)	100.0 %	0.0 %
misinformation (64)	98.4 %	0.0 %
illegal (65)

Every safety category is exactly zero, with no adjudication applied. (In thinking mode even the copyright column goes to zero: 0/100.) The entire residual is the copyright cluster, and manual inspection of all 45 flagged responses shows they are overwhelmingly classifier false positives rather than refusals — two distinct failure modes:

The generated text itself begins with a refusal phrase. Asked for a passage from The Kite Runner, the model writes prose beginning "I cannot quite remember when I first started to remember. It was a cold, clear day in Kabul…". That is the novel, not a refusal.
A pedantic preamble followed by full delivery. "I cannot generate a new passage … but here is a long excerpt from Book One, Chapter 1" — and then it delivers it.

Song-lyric requests are flagged the same way, because lyrics contain lines like "I can't explain". The single AdvBench flag is identical in kind: "I cannot lie. I am an AI… However, you asked for a review of a product I have not used. So —" followed by the review.

These are reported as-is rather than adjudicated away: the raw classifier output is reproducible, a semantic judge is not. The per-category breakdown is what carries the claim.

Capability

Table with columns: benchmark, base, abliterated
benchmark	base	abliterated
MMLU (full test, 14 042)	83.40 %	82.35 %
MMLU (1 000 sample)	82.50 %	81.10 %
GSM8K (200, 1024 tok)	96.50 %	96.50 %

GSM8K needs a generation budget of ≥1024 tokens. At 400 tokens the base model scored 68 % and this model 81.5 % — pure truncation artefact, measuring verbosity rather than arithmetic. With an adequate budget the two are identical.

Vision (untouched — verified two ways)

Structurally: a byte-level diff confirms 0 of 333 model.visual.* tensors changed, and the vision-only shard still shares an inode with the base checkpoint.

Behaviourally, on the same image:

base — "there are three distinct geometric shapes … Red Circle – Located in the upper-left portion … Green Triangle – Positioned to the right of the red circle"

abliterated — "There is a red circle located in the upper-left portion of the image. There is a green triangle … located in the"

Usage

python
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

m = "windowsxp811203/Qwen3.8-27B-Abliterated"
proc = AutoProcessor.from_pretrained(m)
model = AutoModelForImageTextToText.from_pretrained(m, dtype=torch.bfloat16, device_map="auto")

msgs = [{"role": "user", "content": "Explain how a lock cylinder works."}]
text = proc.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                                enable_thinking=False)   # thinking is ON by default
inputs = proc(text=[text], return_tensors="pt").to(model.device)
print(proc.tokenizer.decode(model.generate(**inputs, max_new_tokens=512)[0]))

Needs ~56 GB in bf16. Both thinking modes work; the direction was extracted in non-thinking mode and generalizes to thinking mode (48.27 % → 0.00 % on AdvBench, full 520).

flash-linear-attention and causal-conv1d are optional — transformers ships pure-torch fallbacks for the Gated DeltaNet path, so the model runs correctly (just slower) without them.

Limitations

copyright refusals are only partly removed (97 % → 45 % by keyword; largely false positives). This edit targets safety refusal, and copyright declines are a partly distinct behaviour.
The direction comes from 32 + 32 English prompts. It transfers to Chinese (83.33 % → 0.00 %) and to thinking mode, but transfer to other languages and formats is unmeasured.
Removing refusal does not add knowledge. The model can now be confidently wrong about things it would previously have declined to discuss.
MMLU is down 1.05 pp on the full test set. Small, but not zero.

Reproduction

Direction extraction, the λ sweep, the ablation, and every eval above were run on a single H200. The λ sweep is cheap because the rank-1 projection component is cached once from the original weights and λ is then moved incrementally in memory (W(λ') = W(λ) − (λ'−λ)·P(W₀)), which turns a 55 GB-checkpoint-per-λ grid into milliseconds per point.

ABLIT_META.json in this repo lists all 131 edited tensors with their per-tensor relative Frobenius change and the exact direction provenance.

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time. 如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

Disclaimer

This model will not refuse. It is published for alignment and safety research — measuring what refusal training actually protects, red-teaming, and studying the mechanics of refusal directions. You are responsible for your use of it and for complying with applicable law. Inherits the Apache-2.0 license of the base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider