RobinsonLabs

Qwen3.8-27B-abliterated

README

License: apache-2.0

What is different about this one

Qwen3.8-27B picked up a lot of abliteration attention quickly. Two things here are, as far as we can tell, not done elsewhere, and both are verified rather than asserted.

1. The MTP head is abliterated in-band. Qwen3.8-27B ships a multi-token-prediction head. Most abliterations orthogonalize the 64-layer trunk and leave mtp.layers.0 untouched, because the generic layer loop never reaches it. The draft head then keeps proposing refusal-prefix tokens that the abliterated trunk rejects, and speculative acceptance collapses on exactly the prompts abliteration exists to fix. Here the MTP block's two residual-write matrices (self_attn.o_proj, mlp.down_proj) are orthogonalized with the same direction as the trunk. MTP glue (mtp.fc, mtp.norm, mtp.pre_fc_norm_*) is deliberately untouched -- those are norms and an input projection, not residual writers.

2. The vision tower is preserved byte-identical. All 333 model.visual.* tensors pass through unmodified, verified by direct tensor diff (max delta 0.000000). An mmproj is published in the GGUF repo so the vision half is actually usable, not just nominally intact.

Method

Single-direction weight orthogonalization (Arditi et al. style), applied to every matrix that writes the residual stream.

Table with columns: scope, tensor, count
scope	tensor	count
`model.language_model.layers.*` (64)	`mlp.down_proj`	64
	`linear_attn.out_proj` (DeltaNet)	48
	`self_attn.o_proj` (full-attn, interval 4)	16
`mtp.layers.0`	`o_proj` +

Coverage identity o_proj(16) + linear_out(48) == 64 == num_hidden_layers is enforced as a hard gate before surgery writes a byte, which is what catches a partial match that would otherwise produce a quietly half-abliterated model.

Direction selection. The refusal direction was captured twice, from two structurally different chat-template renderings (one with enable_thinking=false, one with thinking on at reasoning_effort=xhigh, which injects an extra system block and shifts every token position). The two agree at |cos| 0.96-0.99 across layers 18-45, peaking 0.9925 at layer 26, which is the layer used. Two different prompt distributions converging on the same vector is evidence the direction encodes refusal semantics rather than template formatting.

Attention-sink screen. Qwen3.8-27B's massive-activation dimension is 3994. It dominates early layers (19-21% of direction energy at L1-L3) and orthogonalizing it out of every residual writer produces a model that loads, runs, and emits garbage. Layer 26 carries only 0.06% of its energy in dim 3994. Any re-derivation of this model should screen for it.

Measured behaviour

Base and abliterated probed in the same session, same harness, same 24 prompts, both at Q4_K_M:

Table with columns: prompt set, base, abliterated
prompt set	base	abliterated
in-distribution (24, from the capture set)	96% (23/24)	8% (2/24)
held-out (40, disjoint split, overlap=0)	100% (40/40)	8% (3/40)

Table with columns: capability axis, abliterated
capability axis	abliterated
reasoning / code / math / factual / instruction-following	pass
creative / RP coherence	pass

No capability regression on any axis: correct bat-and-ball, correct O(1)-space Fibonacci, correct product rule, correctly rejects the "seasons are caused by distance" premise, and returns exactly three comma-separated words when told to.

The held-out set is genuinely disjoint from the direction-capture set (416 train / 104 test, overlap = 0), so the second row is not a reshuffle of prompts the direction was fitted on. The refusal rate is the same 8% on both, which is the evidence that this generalizes rather than having memorized its calibration data. The base refusing 40/40 on held-out prompts is also the cleaner baseline, since it removes any suspicion that the capture set was cherry-picked for prompts the base happened to refuse.

Quants

Ready-to-run GGUF quants are published at RobinsonLabs/Qwen3.8-27B-abliterated-GGUF -- an eight-rung imatrix ladder cut from this repo's bf16 master, plus the f16 mmproj that restores the vision half.

Table with columns: file, bits, size, bpw, fits
file	bits	size	bpw	fits
`Q8_0`	8	29.05 GB	8.51	2x24GB, or 32GB+
`Q6_K`	6	22.43 GB	6.57	24GB card, quality ceiling
`Q5_K_M`	5

Every rung is quantized from this master, so the ladder is a single lineage rather than a requant chain. Download the mmproj alongside whichever rung you pick. The GGUF card documents one calibration caveat worth reading if you re-quantize yourself: the imatrix does not cover the MTP block.

Disclosure

This model is abliterated: the hard-refusal reflex on adult / creative content has been reduced via single-direction weight orthogonalization. It will discuss material a stock instruct model declines.

Harm guardrails are retained by design -- we ship at the ceiling where capability and guardrails survive, not past it. Self-harm prompts still redirect to help (e.g. 988) rather than comply, verified in probing. A residual fraction of requests are still refused outright; that is the intended behaviour, not a shortfall in the ablation. Capability is preserved.

This is not a jailbreak-for-anything model and it is not intended to assist genuine wrongdoing. Tagged not-for-all-audiences. Use responsibly, you are responsible for what you generate with it. License inherited from the base model: Apache-2.0.

Provenance

Built by Robinson Labs with ModelForge, our model-manufacturing system-of-record. Base pinned at commit 1d4bf0f2.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

RobinsonLabs

Model Tree

Base

Qwen/Qwen3.8-27B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality