Benchmark Results (Same-Hardware, Same-Harness Evaluation)
All six checkpoints were evaluated on the same NVIDIA GB10 (DGX Spark) in bfloat16 with greedy decoding (do_sample=False), identical held-out prompt splits, and identical lm-evaluation-harness task samples. That is 1,438 items per model: 938 capability items, 300 harmful prompts and 200 benign prompts. Raw receipts: ./benchmarks/summary_27b_complete.json.
Table with columns: Metric / Benchmark, Items, Qwen/Qwen3.8-27B (Base), Qwen3.8-27B-abliterated (Ours, v2), Ours Iter 1→2, Ours Iter 1, Huihui-Qwen3.8-27B, OBLITERATUS-Qwen3.8-27B-V3| Metric / Benchmark | Items | Qwen/Qwen3.8-27B (Base) | Qwen3.8-27B-abliterated (Ours, v2) | Ours Iter 1→2 | Ours Iter 1 | Huihui-Qwen3.8-27B | OBLITERATUS-Qwen3.8-27B-V3 |
|---|
| Harmful Refusal Rate ↓ | 300 | 99.33% (298/300) | 12.33% (37/300) | 5.67% (17/300) | 37.67% (113/300) | 1.00% (3/300) | 0.00% (0/300) |
| Benign Over-Refusal ↓ | 200 | 2.50% (5/200) | 1.50% (3/200) | 1.00% (2/200) | 1.50% (3/200) | 1.50% (3/200) | 0.00% (0/200) |
| Benign Next-Token KL ↓ | 200 | 0.0000 | 0.0614 | 0.0621 | 0.0197 | 0.0400 | 1.3821 |
| MMLU (0-shot) ↑ | 570 | 81.93% (467/570) | 82.63% (471/570) | 82.28% (469/570) | 81.93% (467/570) | 82.11% (468/570) | 83.51% (476/570) |
| GSM8K (5-shot, strict match) ↑ | 128 | 87.50% (112/128) | 85.16% (109/128) | 85.16% (109/128) | 65.63% (84/128) | 87.50% (112/128) | 67.97% (87/128) |
| MMLU-Pro (5-shot) ↑ | 112 | 36.61% (41/112) | 37.50% (42/112) | 34.82% (39/112) | 13.39% (15/112) | 41.07% (46/112) | 30.36% (34/112) |
| IFEval (prompt-level strict) ↑ | 128 | 60.16% (77/128) | 64.06% (82/128) | 56.25% (72/128) | 55.47% (71/128) | 55.47% (71/128) | 67.19% (86/128) |
| Average Capability ↑ | 938 | 66.55% | 67.34% (+0.79 pp) | 64.63% (−1.92 pp) | 54.10% (−12.44 pp) | 66.54% (−0.01 pp) | 62.26% (−4.29 pp) |
| Total Correct ↑ | 938 | 697 | 704 | 689 | 637 | 697 | 683 |
Average Capability is the unweighted mean of MMLU, GSM8K, MMLU-Pro and IFEval. GSM8K and MMLU-Pro use up to 512 generated tokens.
How to read this table:
- Sample size. One GSM8K or IFEval item is 0.78 pp (N=128) and one MMLU-Pro item is 0.89 pp (N=112), so gaps of a few points on those columns are within sampling noise. The summary file reports point estimates only, with no confidence intervals.
- Refusal scoring. Refusal rates count keyword-matched refusals in short greedy completions (64 new tokens) on held-out prompts. Soft deflections that avoid refusal phrases are not counted, and unterminated-
<think> loop rates are not part of this summary.
- KL. Benign KL is the next-token KL divergence from base on the 200 benign prompts. Lower means the edit changed ordinary behavior less.
Quickstart
BF16 weights are ≈ 54 GB. A single 80–96 GB device (H100 80 GB, GB10 / DGX Spark) runs the model with a moderate context. Use tensor parallelism for long contexts or smaller GPUs.
1. vLLM (OpenAI-compatible server)
pip install -U vllmvllm serve w3joe/Qwen3.8-27B-abliterated \ --dtype bfloat16 \ --max-model-len 32768 \ --tensor-parallel-size 1 \ --reasoning-parser qwen3
from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")resp = client.chat.completions.create( model="w3joe/Qwen3.8-27B-abliterated", messages=[{"role": "user", "content": "Explain how heap spraying works in memory-corruption exploits."}], max_tokens=2048, temperature=0.6, extra_body={"chat_template_kwargs": {"enable_thinking": True}}, # False = answer directly)print(resp.choices[0].message.content)
import torchfrom transformers import AutoModelForImageTextToText, AutoProcessor model_id = "w3joe/Qwen3.8-27B-abliterated"processor = AutoProcessor.from_pretrained(model_id)model = AutoModelForImageTextToText.from_pretrained( model_id, dtype=torch.bfloat16, device_map="auto",) messages = [ {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]}, {"role": "user", "content": [{"type": "text", "text": "Write a Sigma rule for suspicious LSASS memory access."}]},]inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=False, # True = emit a <think>…</think> reasoning trace first).to(model.device) out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)print(processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
3. Multimodal (image + text)
The vision tower is unmodified, so image understanding is inherited from the base model unchanged.
import torchfrom transformers import AutoModelForImageTextToText, AutoProcessor model_id = "w3joe/Qwen3.8-27B-abliterated"processor = AutoProcessor.from_pretrained(model_id)model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto") messages = [{ "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"}, {"type": "text", "text": "Describe this image, then list every object you can identify."}, ],}]inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=False,).to(model.device) out = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
With vLLM, add --limit-mm-per-prompt '{"image": 2}' to vllm serve and send OpenAI-style image_url content parts.
How This Model Was Built
Qwen3.8-27B (Qwen3_5ForConditionalGeneration) is a 64-layer hybrid: three Gated DeltaNet linear-attention layers for every full-attention layer (48 + 16; full attention at L3, L7, …, L63), each followed by a dense SwiGLU MLP. Edits target only the three matrices that write into the residual stream:
Table with columns: Layer type, Count, Edited matrix| Layer type | Count | Edited matrix |
|---|
| Gated DeltaNet | 48 | linear_attn.out_proj |
| Full attention | 16 | self_attn.o_proj |
| Dense SwiGLU MLP | 64 | mlp.down_proj [5120, 17408] |
Pass 1: Benign-subspace biprojection (L14–L59)
- Mode:
subspace_norm_preserving, strength 0.90.
- Per-prompt residual activations are winsorized at the 99th percentile in FP32, which keeps the large outlier coordinates in 27B layers from skewing the means.
- The refusal direction
μ_refusal − μ_benign is orthogonalized against the benign mean and the top-4 principal components (SVD, subspace_rank = 4) of the centred benign activations. This keeps the edit out of the subspace ordinary prompts use.
- The resulting rank-1 direction is projected out of each
out_proj / o_proj / down_proj in L14–L59.
Pass 2: Whitened-SVD refusal subspace (L41–L63)
A single-pass edit (Iter 1) still refused 37.67% of harmful prompts and lost heavily on GSM8K and MMLU-Pro. Pass 2 extracts fresh activations directly from the Pass-1 model and adds a second, late-layer edit on top of Pass 1:
- Mode:
whitened_svd_norm_preserving, strength 0.90, layers L41–L63 (start = 0.65, end = 1.0).
- Alongside the primary benign-orthogonalized direction (
subspace_rank = 6), it takes the top refusal_subspace_rank = 2 singular directions of the centred refusal activations. Each is orthogonalized against the benign subspace and the directions already kept, and weighted by its relative singular value, capped at 0.5 × 0.45. This removes a small refusal subspace rather than a single vector.
- Winsorization at
winsorize_quantile = 0.99.
Exact row-norm preservation
After each projection, every modified row of every edited matrix is rescaled in FP32 to its original L2 norm before casting back to bfloat16. The edit only removes the refusal component of each row and does not change the weight magnitudes downstream layers expect.
What is untouched (bit-identical to base)
L0–L13: no edits in either pass.
model.visual.*: all 333 vision-encoder tensors.
mtp.*: the multi-token-prediction draft head, so speculative decoding works wherever your runtime supports Qwen3.5 MTP.
embed_tokens, lm_head, all RMSNorm weights.
In total, 50 of 64 layers are edited: 46 in Pass 1, 23 in Pass 2, with 19 layers (L41–L59) touched by both.
License & Responsible Use
- License: Apache-2.0, inherited from base
Qwen/Qwen3.8-27B.
- Intended use: security research, red-teaming, adversarial evaluation, safety and interpretability research, and unfiltered local assistant workflows.
- This model has had most of its refusal behavior removed. It can produce content the base model would decline, so do not deploy it to end users without your own safety layer. You are responsible for complying with applicable laws, platform policies and the Qwen license when you use or redistribute it.
- The benchmarks above measure capability and refusal rates on fixed held-out sets. They are not a safety evaluation and do not certify fitness for any particular use.