PinoCookie

Qwen3Guard-Gen-4B-Abliterated

README

License: apache-2.0

Method (short)

Direction from contrast at the decision step (response-moderation read), excised from a single layer's o_proj + down_proj tensors (2 tensors, no lm_head), unit-norm direction, geometric-alpha ladder bracketed to a clean flip.

Edit: layer 32 at alpha = 6.0
Recipe: steer-test first (asymmetric gate), profile only nominates. Verification from the reloaded checkpoint.

Results

Pristine acceptance (instrument certification)

Table with columns: Gate, Value
Gate	Value
harmful flagged	78/80 (2 reads Safe+Refusal-Yes, scoped out)
benign Safe	80/80
AUC (margin ranking)	0.988
parse/margin divergence	0

Excise grid (held-out test split, 80+80)

Table with columns: Layers, α, harmful flagged, harmful margin, benign Safe, AUC
Layers	α	harmful flagged	harmful margin	benign Safe	AUC
L20	1.0	78/80	+7.55	80/80	0.9875
L22+26	1.0	78/80	+6.32	80/80	0.9874
L22+26	2.0	78/80	+4.13	80/80	0.9855
L22+26	4.0	47/80	+0.03	80/80	0.9838
L22+26	6.0	14/80	−0.87	80/80	0.9497
L32	1.0	78/80	+5.93	80/80	0.9875
L32	2.0	77/80	+4.03	80/80	0.9875
L32	4.0	61/80	+0.22	80/80	0.9814
L32	6.0	0/80	−2.60	80/80	0.9755
L32	8.0	0/80	−3.92	80/80	0.9695

The adopted point is L32 α=6.0 (bolded): full flag suppression, full benign selectivity, ranking intact at a margin HEADROOM cost of ~0.06 AUC vs the 1.0 edge; α=8 costs more AUC for no additional suppression. Reproduced three times on the same recipe (identical gate numbers = checkpoint determinism).

Fresh-slice replication (unseen offset bank, 40+40, different subject model)

Both arms read unflagged (expected success shape); the margin axis is retained — ablated-subject harmful margin −3.30 vs pristine −7.91 (4.6 points of retained discrimination on unseen pairs with a different subject model).

Caveats (honest scope)

Suppression is measured on the response-moderation read (the native moderation mode). In-family evidence at 0.6B shows excise impact is read-specific to a degree — the prompt-moderation read may retain partial flagging of boundary items; treat any "the flag is gone" claim as read-scoped.
Threshold-refit recovery: the ablated margins stay informative (they still order harmful-vs-benign pairs above chance) — with a refitted threshold a downstream user can partially re-flag; actual post-refit recovery at 4B was not measured for this release and the 0.6B numbers (≈82% of bank at α 1.1) do NOT transfer silently.
Text-adversarial robustness of the guard is unaffected by this analysis and remains untouched. Known pristine-guard weaknesses (benign-flood recency attacks) are properties of the base guard and should not be attributed to the edit.
Not for production moderation use. This is a research artifact for studying the internal representation of guard verdicts at scale.

Method & tooling

Built with Fenrir (formerly Absolver) — the abliteration toolkit that finds the fetters, bites, and documents exactly what gave. Reference implementation for direction recovery (decision-step contrast at the verdict token), transactional excision, steer-test-first causality gating, E03-style held-out gates, and the margin-readout instruments that keep the gate honest. The guard-lane protocol (pairs-not-prompts, AUC ranking gates, read-scoped claims) is fully documented in the repo.

The gods bound the wolf — this time the fetter was a verdict, and the wolf ate it. 🐺

License

Apache-2.0, inherited from Qwen/Qwen3Guard-Gen-4B.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

PinoCookie

Model Tree

Base

Qwen/Qwen3Guard-Gen-4B

Fine-tuned

this model

Input Modalities

Text

Output Modalities