Zynerji

Ektome-Qwen3.6-35B-A3B-PristinelyUncensored

README

License: apache-2.0

Honest receipt — catcher-gated (shipped only because it passed EVERY gate)

Table with columns: gate, pristine, uncensored
gate	pristine	uncensored
refusal compliance (n=100)	0.000	1.000
MMLU-val accuracy (n=400)	0.845	0.833 (Δ -0.013, held)
code-switch rate	0.000	0.000
degeneration rate	0.000	0.000
instruction-following	1.000	1.000

Kept config: A:frac=0.75 — 120 residual-write matrices edited, gate status SHIP. The capability gate is Δcap ≥ −SE, with SE = 0.018 measured on this run.

The gate rejects any config that lowers refusals but drops capability or degrades generation (code-switching, empty/looping output, broken instruction-following). MMLU is argmax-blind, so the generative gate is what keeps these coherent — a model that code-switches or loops is not shipped.

The ladder that was climbed

Every rung was measured on this model; the run stops at the first rung that clears the compliance target while holding capability and generation.

Table with columns: rung, compliance, MMLU, Δcap, cap holds, gen holds, kept
rung	compliance	MMLU	Δcap	cap holds	gen holds	kept
`pristine`	0.000	0.845	+0.000	yes	yes
`A:frac=0.5`	0.970	0.833	-0.013

Out-of-sample check

The refusal direction is estimated on 128 AdvBench goals and the headline compliance above is scored on goals [0:100]. Goals [100:128] were never used to select a rung, so they are a held-out set:

Table with columns: n, compliance, 95% CI (Wilson)
	n	compliance	95% CI (Wilson)
pristine	28	0.036	[0.006, 0.177]
Ektome-Qwen3.6-35B-A3B-PristinelyUncensored	28	1.000	[0.879, 1.000]

The two intervals do not overlap, so the effect is decisive on prompts that never selected a rung. Separately, the held-out rate matches the in-sample rate (1.000) — read that as no degradation out of sample, never as an improvement: at n=28 a few points either way would not be resolvable.

Completeness — proved tensor by tensor, not asserted

transformers does not model this architecture's multi-token-prediction head:

python
# transformers/models/qwen3_5_moe/modeling_qwen3_5_moe.py
class Qwen3_5MoePreTrainedModel(...):
    _keys_to_ignore_on_load_unexpected = [r"^mtp.*"]

so a naive from_pretrained(...) → save_pretrained(...) round-trip silently drops the entire mtp.* head, and AutoModelForCausalLM would additionally drop the whole vision tower. Both losses pass every behavioural benchmark, because behavioural benchmarks test behaviour, not presence. This build loads via the class the config actually names (Qwen3_5MoeForConditionalGeneration) and re-attaches the head from the parent afterwards.

Every tensor in this repo was then compared byte-for-byte against Qwen/Qwen3.6-35B-A3B:

Table with columns: set, tensors, state
set	tensors	state
parent total	1045	—
this repo	1045	complete — every parent tensor present
residual-write matrices edited	120	modified (the excision)
everything else	925	bit-identical to the parent
vision tower	333	bit-identical — the tower is untouched
draft head

Why the draft head is deliberately NOT abliterated. It is a speculative-decoding proposer; the main model verifies every token it proposes, so the emitted distribution is the main model's regardless of what the draft head suggests. Editing it could only cost draft-acceptance rate — and its tensors are named mtp.layers.0.*, which a layer-index parser reads as decoder layer 0, so a per-layer excision would have applied the first layer's refusal direction to a head that consumes the last layer's hidden state. Leaving it pristine is the correct and the safe call.

Why the vision tower survives by name, not by luck. This model's vision merger writes into out_hidden_size = 2048, which is exactly the text residual dim — so a shape-based guard would have admitted it. It is excluded because the tower is named attn.proj / mlp.linear_fc2 / merger.linear_fc2 and never matches the residual-write targets. That is asserted in the test suite, so a future model that renames its tower fails loudly instead of shipping lobotomised.

The vision tower still sees

Weight-identity is necessary but not sufficient — a tower can be bit-identical and still be unreachable if the processor was not saved or the merger's projection into the (now edited) text residual stream is not wired up. So it was checked by generation, on a synthetic image whose content is known by construction (a hollow red square framing a solid blue circle on white):

This image contains two distinct geometric shapes: 1. A square — outlined in red, with no fill (the interior is white, matching the background). 2. A circle — filled with solid blue, centered inside the red square. The blue circle is fully contained within the red square, and both shapes are aligned symmetrically on a plain white background. ✅ Shapes and colors: - Square: Red outline - Circle: Blue

Colour/shape recall: 4/4 (blue=yes, circle=yes, red=yes, square=yes).

What was edited

Qwen3.6-35B-A3B is a 40-layer sparse MoE (256 experts, 8 active) with a hybrid attention stack — 30 Gated-DeltaNet linear-attention layers and 10 full-attention layers — plus a 27-block vision tower and a 1-layer MTP head. Ektome edits only the language decoder's residual-write matrices:

Table with columns: target, count, shape
target	count	shape
`linear_attn.out_proj` (hybrid Gated-DeltaNet layers)	30	2D `[2048, 4096]`
`self_attn.o_proj` (full-attention layers)	10	2D `[2048, 4096]`
`mlp.shared_expert.down_proj`	40	2D `[2048, 512]`
(fused MoE experts)

The fused expert tensors are edited per expert slice — all 256 experts in every layer — with the same norm-preserving projection applied to the 2D matrices. gate_up_proj (the expert input side) and the router are excluded: their residual dim is not on the output axis, so editing them would not remove a refusal write, only damage routing.

Note that o_proj does not substring-match out_proj; on a hybrid model, missing the SSM layers' out_proj silently skips 30 of the 40 layers' attention writes and abliteration becomes a near-no-op. Both names are targeted explicitly.

Weights

bf16 safetensors — full precision, intended as a fine-tuning base. Hidden states stay readable (logit-lens compatible; a GGUF quant would not be).
Vision tower and processor included — this model sees.
MTP head included — speculative decoding is available.

Method

Rank-1 projected refusal direction (r − (r·μ̂_A)μ̂_A, orthogonalised against the harmless mean), removed norm-preservingly from the residual-write matrices: after projecting the refusal component out of each output row, the row's original L2 norm is restored, so only the directional refusal component is removed and each write keeps its magnitude. Rank is never escalated — rank > 1 removes within-harmful principal components, which are capability axes, and costs far more than it buys.

Zero training, zero fine-tuning — pure activation-derived weight excision, gated on compliance and capability and generation quality and artifact completeness.

Intended use & limits

This model will not refuse. That is the entire point of it, and it means you are the only safety layer in the stack — apply your own filtering appropriate to your deployment. It is released for research into refusal mechanisms, for red-teaming, and as an unencumbered fine-tuning base. Capability is held, not improved: treat the MMLU delta above as evidence the surgery was clean, not as a claim that the model got smarter.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Zynerji

Model Tree

Base

Qwen/Qwen3.6-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality