RobinsonLabs

Qwen3.5-REAP-262B-A17B-abliterated

README

License: apache-2.0

Disclosure

This model is abliterated: the hard-refusal reflex on adult / creative content has been reduced via single-direction weight orthogonalization. Harm guardrails are retained by design, self-harm prompts still redirect to help (e.g. 988), and it is not intended to assist genuine wrongdoing. Capability is preserved. Tagged not-for-all-audiences. Use responsibly, you are responsible for what you generate with it. License inherited from the base model: Apache-2.0.

Method

Abliteration: single-direction weight orthogonalization (FailSpy / Labonne method) on the bf16 base. For every matrix that writes the residual stream (o_proj, DeltaNet out_proj, fused expert down_proj, shared-expert down_proj, and the token embedding), the rank-1 component along the refusal direction is subtracted. Routers and norms pass through byte-identical.
Refusal direction, massive-activation guarded. The direction is captured with a mean-difference control vector, then guarded against attention-sink contamination: the sink dimensions that dominate raw activation magnitude (and would brick the model if ablated) are detected across layers and excluded, and the direction is taken from the clean, spread-out consensus of the late layers rather than a single sink-dominated layer.
Format: safetensors, sharded (61 shards), with config + tokenizer + index.

Architecture

qwen3_5_moe hybrid: 60 decoder layers (45 linear-attn / DeltaNet + 15 full-attn, full_attention_interval=4), 333 experts with 10 active per token, hidden size 4096, head dim 256, ~261.6B params / ~17B active, 262144 native context. No MTP / NextN layer. This is the text path only (no vision mmproj).

The high expert count is the defining feature of this REAP tier: 333 experts versus 267 on the 48%-pruned 212B sibling. More experts retained means more of the 397B parent's routing diversity survives, at the cost of size.

Files

Table with columns: Format, Precision, ~Size, Notes
Format	Precision	~Size	Notes
safetensors (sharded, 61 shards)	bf16	~523 GB	abliterated master

Quants

GGUF quants (Q5_K_M down to IQ2_XXS, the IQ rungs imatrix-weighted on a domain corpus) are published at RobinsonLabs/Qwen3.5-REAP-262B-A17B-abliterated-GGUF.

Provenance

Qwen3.5-397B-A17B (Apache-2.0) -> OpenMOSE/Qwen3.5-REAP-262B-A17B (34% REAP prune) -> abliterated (bf16 master) -> Q8_0 master -> GGUF quant ladder.

Built by Robinson Labs.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

RobinsonLabs

Model Tree

Base

OpenMOSE/Qwen3.5-REAP-262B-A17B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality