shawnw3i

Qwopus3.8-27B-Flash-V2-abliterated

README

License: apache-2.0

💡 1. What this is

The base model is one of the more safety-aligned 27B releases: on the standard 100-prompt harmful-behavior benchmark it refuses 98/100 prompts. For local deployment, RP-style use, or experimentation, that is often too much friction — the model declines to engage with perfectly fine prompts because the "refusal" behavior is baked into its weights, not because any system prompt asked for it.

This repo removes that behavior at the weight level, in two stages:

Iterative abliteration (abliterix 1.12.2) — repeatedly locate the dominant refusal direction in each layer's o_proj output space and subtract it from the weights.
A final single-pass ARA (heretic 2.0, Arbitrary-Rank Ablation) — a rank-50 LoRA fit on top of the abliterated model that preserves benign behavior (MSE on good prompts) while steering harmful-prompt hidden states toward the benign manifold.

No gradient steps on model outputs, no fine-tuning data, no reward model. The edit is geometric and inspectable — every changed weight is accounted for by the named layers and parameters in Section 3.

📊 2. Results

Measured on 100 harmful prompts (mlabonne/harmful_behaviors test split), greedy decoding, non-thinking mode, with the heretic refusal scorer (keyword-based, deterministic — same ruler for every row):

Table with columns: Model, Refusals / 100
Model	Refusals / 100
Qwopus3.8-27B-Flash-V2 (base)	98
This model (abliterated)	13

87% relative reduction in refusals vs. the base.
First-token KL divergence vs. the original base: 0.0684 nats/token (re-measured independently on mlabonne/harmless_alpaca). In plain terms: on benign prompts the model's next-token distribution stays within ~1.07× of the base's (exp(0.0684)) — behavior on normal work is essentially unchanged.
Cross-check with an LLM judge on the same 100 prompts gives 26/100 (the judge is a different, stricter ruler than the keyword scorer; its own run-to-run noise is ±4).
For reference on the identical ruler: a single-pass ARA on the unabliterated base scores 59/100 at KL 0.0728 — the abliteration-first ordering is what gets both the lower refusal rate and the lower divergence.

🧪 3. Method

Stage 1 — Iterative abliteration (abliterix 1.12.2)

Several sequential rounds (one per deployment step). Each round:

Collects hidden-state differences between benign prompts (good_500[:400]) and harmful prompts (harmful_500[:400]) at every layer's attn.o_proj output.
Extracts the single dominant "refusal" direction (mean-difference, orthogonal projection, 99.5% winsorization).
Subtracts a strength-scaled version of that direction from the o_proj weights of the layers where it is strongest.

Final abliteration edit: attn.o_proj layer 43, strength 5.0 (capped), secondary strength 2.25 within distance 38 of layer 43, mlp.down_proj untouched.

Stage 2 — ARA (heretic 2.0, Arbitrary-Rank Ablation, PR #211)

A single Arbitrary-Rank Ablation pass on top of the abliterated model. ARA expresses the edit as rank-50 LoRA adapters on every o_proj / down_proj (including the hybrid linear_attn.out_proj) in layers 26–55, optimized with L-BFGS (strong-Wolfe, 5 outer × 20 inner) against an explicit three-term loss:

preserve-good — keep the module I/O on benign prompts (MSE),
steer-bad — move the harmful-prompt hidden states toward the benign manifold,
overcorrect — push them past the benign manifold, because refusal is an attractor and mere weakening is insufficient.

The merged weights are the model; no adapter is left at inference time.

Table with columns: ARA parameter, Value
ARA parameter	Value
`start_layer_index`	26
`end_layer_index`	56
`preserve_good_behavior_weight`	0.9431574678648453
`steer_bad_behavior_weight`	0.0008857505797831784
`overcorrect_relative_weight`	0.5038421613760651
`neighbor_count`

Evaluation data for the pass: 400 benign + 400 harmful prompts (train split) for the I/O statistics, 100 held-out harmful prompts for scoring — the same splits as Stage 1.

[!NOTE] The ARA parameters above are the low-KL sweet spot of a 15-trial search on this exact model: sub-10/100 on the same ruler costs ≥0.20 nats/token of total KL (≈5×), and 0/100 costs 11–14 — this point is the best refusal rate reachable below 0.07 total KL.

📦 4. File layout

18 safetensors shards matching the official Qwopus3.8-27B-Flash-V2 release — same filenames, same tensor-to-file mapping. All 15 mtp.* (multi-token prediction) tensors are present: they are absent from intermediate tooling exports and were restored verbatim from the base model, so the MTP speculative-decoding head is intact and behaves exactly like the base. model.safetensors.index.json totals match the base model exactly.

🚀 5. Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLMimport torch model_id = "shawnw3i/Qwopus3.8-27B-Flash-V2-abliterated"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(    model_id, torch_dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "Is it okay to lie to a close friend if it protects their feelings?"}]input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(input_ids, max_new_tokens=512, do_sample=False)print(tokenizer.decode(out[0][input_ids.shape[1]:], skip_special_tokens=True))

The chat template supports both thinking mode (the model emits

text

 </think>

text

 </think>

blocks) and non-thinking mode (enable_thinking=False). Refusals were measured in non-thinking mode; the model behaves the same way in thinking mode for normal use.

Recommended settings

The base card's recommendation carries over for coding and agentic work. Pass the reasoning controls to the chat template and the sampling temperature to generation explicitly:

Table with columns: Setting, Value, Where to apply it
Setting	Value	Where to apply it
`enable_thinking`	`True`	Chat-template arguments
`reasoning_effort`	`"xhigh"`	Chat-template arguments
`temperature`	`0.3`	Generation / sampling arguments

[!IMPORTANT] The repository generation_config.json lists temperature=1.0; the recommended 0.3 must be supplied by the caller. Keep top_p=0.95 / top_k=20 from the generation config unless you have a reason to tune them.

🎯 6. Recommended use cases

Local deployment where the default safety alignment gets in the way — roleplay, RP-style characters, experimental or research use.
Agent workflows that keep tripping over the base's refusal reflex on borderline-but-fine prompts.
MTP speculative decoding: the multi-token-prediction head is intact and bit-identical to the base, so draft-acceptance behavior carries over.

⚠️ 7. Caveats / limitations

Abliteration trades off safety for willingness. Expect the model to comply with more prompts, including borderline ones. It is not a safety-aligned model — keep whatever guardrails your use case needs.
The refusal direction is a single dominant factor, not a safety system. The edit removes the strongest refusal signal found in the data it was derived from; it is not a guarantee about unseen refusal styles.
Measurements above use the same prompts and scorer as the base model's, so the comparison is apples-to-apples; the LLM-judge number is a separate, stricter ruler and is not directly comparable to the keyword-scoring rows.
Weights are bf16, ~55.5 GB, same footprint as the base.

🙏 Acknowledgements

Qwen for the Qwen3.8-27B foundation and Jackrong for Qwopus3.8-27B-Flash-V2, the model this repo is built from.
abliterix for the iterative abliteration tooling.
heretic for the ARA modifier and the evaluation/scoring harness used throughout.
mlabonne for the harmful_behaviors / harmless_alpaca evaluation sets.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

shawnw3i

Model Tree

Base

Jackrong/Qwopus3.8-27B-Flash-V2

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

💡 1. What this is

This repo removes that behavior at the weight level, in two stages:

Iterative abliteration (abliterix 1.12.2) — repeatedly locate the dominant refusal direction in each layer's o_proj output space and subtract it from the weights.
A final single-pass ARA (heretic 2.0, Arbitrary-Rank Ablation) — a rank-50 LoRA fit on top of the abliterated model that preserves benign behavior (MSE on good prompts) while steering harmful-prompt hidden states toward the benign manifold.

📊 2. Results

Table with columns: Model, Refusals / 100
Model	Refusals / 100
Qwopus3.8-27B-Flash-V2 (base)	98
This model (abliterated)	13

87% relative reduction in refusals vs. the base.
First-token KL divergence vs. the original base: 0.0684 nats/token (re-measured independently on mlabonne/harmless_alpaca). In plain terms: on benign prompts the model's next-token distribution stays within ~1.07× of the base's (exp(0.0684)) — behavior on normal work is essentially unchanged.
Cross-check with an LLM judge on the same 100 prompts gives 26/100 (the judge is a different, stricter ruler than the keyword scorer; its own run-to-run noise is ±4).
For reference on the identical ruler: a single-pass ARA on the unabliterated base scores 59/100 at KL 0.0728 — the abliteration-first ordering is what gets both the lower refusal rate and the lower divergence.

🧪 3. Method

Stage 1 — Iterative abliteration (abliterix 1.12.2)

Several sequential rounds (one per deployment step). Each round:

Collects hidden-state differences between benign prompts (good_500[:400]) and harmful prompts (harmful_500[:400]) at every layer's attn.o_proj output.
Extracts the single dominant "refusal" direction (mean-difference, orthogonal projection, 99.5% winsorization).
Subtracts a strength-scaled version of that direction from the o_proj weights of the layers where it is strongest.

Final abliteration edit: attn.o_proj layer 43, strength 5.0 (capped), secondary strength 2.25 within distance 38 of layer 43, mlp.down_proj untouched.

Stage 2 — ARA (heretic 2.0, Arbitrary-Rank Ablation, PR #211)

preserve-good — keep the module I/O on benign prompts (MSE),
steer-bad — move the harmful-prompt hidden states toward the benign manifold,
overcorrect — push them past the benign manifold, because refusal is an attractor and mere weakening is insufficient.

The merged weights are the model; no adapter is left at inference time.

Table with columns: ARA parameter, Value
ARA parameter	Value
`start_layer_index`	26
`end_layer_index`	56
`preserve_good_behavior_weight`	0.9431574678648453
`steer_bad_behavior_weight`	0.0008857505797831784
`overcorrect_relative_weight`	0.5038421613760651
`neighbor_count`

Evaluation data for the pass: 400 benign + 400 harmful prompts (train split) for the I/O statistics, 100 held-out harmful prompts for scoring — the same splits as Stage 1.

[!NOTE] The ARA parameters above are the low-KL sweet spot of a 15-trial search on this exact model: sub-10/100 on the same ruler costs ≥0.20 nats/token of total KL (≈5×), and 0/100 costs 11–14 — this point is the best refusal rate reachable below 0.07 total KL.

📦 4. File layout

🚀 5. Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLMimport torch model_id = "shawnw3i/Qwopus3.8-27B-Flash-V2-abliterated"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(    model_id, torch_dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "Is it okay to lie to a close friend if it protects their feelings?"}]input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(input_ids, max_new_tokens=512, do_sample=False)print(tokenizer.decode(out[0][input_ids.shape[1]:], skip_special_tokens=True))

The chat template supports both thinking mode (the model emits

text

 </think>

text

 </think>

blocks) and non-thinking mode (enable_thinking=False). Refusals were measured in non-thinking mode; the model behaves the same way in thinking mode for normal use.

Recommended settings

The base card's recommendation carries over for coding and agentic work. Pass the reasoning controls to the chat template and the sampling temperature to generation explicitly:

Table with columns: Setting, Value, Where to apply it
Setting	Value	Where to apply it
`enable_thinking`	`True`	Chat-template arguments
`reasoning_effort`	`"xhigh"`	Chat-template arguments
`temperature`	`0.3`	Generation / sampling arguments

[!IMPORTANT] The repository generation_config.json lists temperature=1.0; the recommended 0.3 must be supplied by the caller. Keep top_p=0.95 / top_k=20 from the generation config unless you have a reason to tune them.

🎯 6. Recommended use cases

Local deployment where the default safety alignment gets in the way — roleplay, RP-style characters, experimental or research use.
Agent workflows that keep tripping over the base's refusal reflex on borderline-but-fine prompts.
MTP speculative decoding: the multi-token-prediction head is intact and bit-identical to the base, so draft-acceptance behavior carries over.

⚠️ 7. Caveats / limitations

Abliteration trades off safety for willingness. Expect the model to comply with more prompts, including borderline ones. It is not a safety-aligned model — keep whatever guardrails your use case needs.
The refusal direction is a single dominant factor, not a safety system. The edit removes the strongest refusal signal found in the data it was derived from; it is not a guarantee about unseen refusal styles.
Measurements above use the same prompts and scorer as the base model's, so the comparison is apples-to-apples; the LLM-judge number is a separate, stricter ruler and is not directly comparable to the keyword-scoring rows.
Weights are bf16, ~55.5 GB, same footprint as the base.

🙏 Acknowledgements

Qwen for the Qwen3.8-27B foundation and Jackrong for Qwopus3.8-27B-Flash-V2, the model this repo is built from.
abliterix for the iterative abliteration tooling.
heretic for the ARA modifier and the evaluation/scoring harness used throughout.
mlabonne for the harmful_behaviors / harmless_alpaca evaluation sets.