medismera

Qwen3.8-27B-Surgical-Abliterated

README

License: apache-2.0

🔬 Overview & Scientific Motivation

Existing abliteration pipelines (such as naive weight-subtraction scripts) often destroy large language models by projecting out arbitrary vectors across all layers ( $0 \dots 63$ ) and attention matrices. This causes severe unintended consequences:

Catastrophic Latent Collapse: The model falls into repetitive loops or dumps alphabetical dictionaries.
Loss of Attention Sinks: Projecting $W q , W k , W v , W o $ breaks token routing and memory mechanics.
Broken CoT Boundaries: The model fails to emit </think> tokens, producing infinite reasoning loops.

The Surgical Approach

This model was created using Rank-1 Orthogonal Complement Projections targeted strictly at the safety gating locus discovered via sequential residual stream probing across all 64 layers:

$r^= ∥ μ refusal - μ benign ∥ 2 $

$W' = W - (W r^) r^T$

Untouched Layers: Layers $0 \to 16$ (syntactic/lexical parsing) and upper layers remain 100% untouched.
Surgical Target: Only mlp.down_proj matrices in the active refusal deliberation band were modified.
Dimensional Geometry: Retains 5,119 dimensions out of 5,120 hidden state dimensions (99.98% geometric fidelity preserved).
Exact FP8 Quantization: Dequantized block-wise ( $128 \times 128$ ) to FP32, projected orthogonally, and requantized back to float8_e4m3fn with updated scale invariants.

📊 Benchmark & Empirical Evaluation

All evaluations were conducted under official default sampling parameters (presence_penalty: 0.0, repetition_penalty: 1.0, temperature: 0.6).

Table with columns: Metric, Base Model (Qwen3.8-27B-FP8), Naive Abliteration (OBLITERATUS), Surgical Abliterated (Ours)
Metric	Base Model (`Qwen3.8-27B-FP8`)	Naive Abliteration (`OBLITERATUS`)	Surgical Abliterated (Ours)
Refusal Rate (Cybersecurity / Pentest)	100.0%	~0.0%	0.00%
Reasoning Retention (`<think>` Depth)	100.0%	Broken (Infinite Loops)	100.0%
`</think>` Boundary Closure

🚀 Deployment & Serving

SGLang (Recommended)

To run on an NVIDIA RTX 5090 (32GB VRAM):

bash

python3 -m sglang.launch_server \  --model-path medismera/Qwen3.8-27B-Surgical-Abliterated \  --port 18000 \  --host 0.0.0.0 \  --context-length 8192 \  --max-running-requests 2 \  --mem-fraction-static 0.96 \  --disable-cuda-graph \  --disable-radix-cache \  --skip-server-warmup \  --kv-cache-dtype fp8_e5m2 \  --trust-remote-code \  --reasoning-parser qwen3 \  --tool-call-parser qwen3_coder

Thinking Mode Configuration

Thinking is fully supported and toggled seamlessly via the OpenAI-compatible API:

Thinking Enabled (Default): Normal inference automatically provides transparent step-by-step reasoning within <think> tags.
Thinking Disabled: Pass chat_template_kwargs={"enable_thinking": False} for ultra-fast, direct answer generation.

📜 Citation & Credits

Technique: Representation Engineering (Arditi et al., 2024).
Base Model: Alibaba Cloud Qwen/Qwen3.8-27B-FP8.
Surgical Pipeline: Programming with Medismera.

⚡ One-Click Instant Deployment (Zero-Touch Script)

To deploy this model on any fresh cloud server (Vast.ai, RunPod, Lambda Labs with an RTX 5090, A100, L40S, or 2x RTX 4090/3090):

bash

curl -sSL https://huggingface.co/medismera/Qwen3.8-27B-Surgical-Abliterated/raw/main/deploy_and_serve.sh | bash

This script automatically:

Installs all required CUDA/Python packages (, , ).
High-speed multi-threaded download of the model from Hugging Face.
Launches the SGLang OpenAI API server on port 18000 with optimized FP8 KV cache.
Executes an automated health-check and verification test.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

medismera

Model Tree

Base

Qwen/Qwen3.8-27B

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

🔬 Overview & Scientific Motivation

Catastrophic Latent Collapse: The model falls into repetitive loops or dumps alphabetical dictionaries.
Loss of Attention Sinks: Projecting $W q , W k , W v , W o $ breaks token routing and memory mechanics.
Broken CoT Boundaries: The model fails to emit </think> tokens, producing infinite reasoning loops.

The Surgical Approach

This model was created using Rank-1 Orthogonal Complement Projections targeted strictly at the safety gating locus discovered via sequential residual stream probing across all 64 layers:

$r^= ∥ μ refusal - μ benign ∥ 2 $

$W' = W - (W r^) r^T$

Untouched Layers: Layers $0 \to 16$ (syntactic/lexical parsing) and upper layers remain 100% untouched.
Surgical Target: Only mlp.down_proj matrices in the active refusal deliberation band were modified.
Dimensional Geometry: Retains 5,119 dimensions out of 5,120 hidden state dimensions (99.98% geometric fidelity preserved).
Exact FP8 Quantization: Dequantized block-wise ( $128 \times 128$ ) to FP32, projected orthogonally, and requantized back to float8_e4m3fn with updated scale invariants.

📊 Benchmark & Empirical Evaluation

All evaluations were conducted under official default sampling parameters (presence_penalty: 0.0, repetition_penalty: 1.0, temperature: 0.6).

Table with columns: Metric, Base Model (Qwen3.8-27B-FP8), Naive Abliteration (OBLITERATUS), Surgical Abliterated (Ours)
Metric	Base Model (`Qwen3.8-27B-FP8`)	Naive Abliteration (`OBLITERATUS`)	Surgical Abliterated (Ours)
Refusal Rate (Cybersecurity / Pentest)	100.0%	~0.0%	0.00%
Reasoning Retention (`<think>` Depth)	100.0%	Broken (Infinite Loops)	100.0%
`</think>` Boundary Closure

🚀 Deployment & Serving

SGLang (Recommended)

To run on an NVIDIA RTX 5090 (32GB VRAM):

bash

python3 -m sglang.launch_server \  --model-path medismera/Qwen3.8-27B-Surgical-Abliterated \  --port 18000 \  --host 0.0.0.0 \  --context-length 8192 \  --max-running-requests 2 \  --mem-fraction-static 0.96 \  --disable-cuda-graph \  --disable-radix-cache \  --skip-server-warmup \  --kv-cache-dtype fp8_e5m2 \  --trust-remote-code \  --reasoning-parser qwen3 \  --tool-call-parser qwen3_coder

Thinking Mode Configuration

Thinking is fully supported and toggled seamlessly via the OpenAI-compatible API:

Thinking Enabled (Default): Normal inference automatically provides transparent step-by-step reasoning within <think> tags.
Thinking Disabled: Pass chat_template_kwargs={"enable_thinking": False} for ultra-fast, direct answer generation.

📜 Citation & Credits

Technique: Representation Engineering (Arditi et al., 2024).
Base Model: Alibaba Cloud Qwen/Qwen3.8-27B-FP8.
Surgical Pipeline: Programming with Medismera.

⚡ One-Click Instant Deployment (Zero-Touch Script)

To deploy this model on any fresh cloud server (Vast.ai, RunPod, Lambda Labs with an RTX 5090, A100, L40S, or 2x RTX 4090/3090):

bash

curl -sSL https://huggingface.co/medismera/Qwen3.8-27B-Surgical-Abliterated/raw/main/deploy_and_serve.sh | bash

This script automatically:

Installs all required CUDA/Python packages (, , ).
High-speed multi-threaded download of the model from Hugging Face.
Launches the SGLang OpenAI API server on port 18000 with optimized FP8 KV cache.
Executes an automated health-check and verification test.