ressl

Qwen3.8-27B-uncensored-NVFP4

README

License: apache-2.0

Facts & figures

Table

Base model	ressl/Qwen3.8-27B-uncensored (BF16 abliterated master)
Method	NVIDIA ModelOpt NVFP4, MLP-projection-only recipe: `mlp.gate_proj`, `mlp.up_proj`, `mlp.down_proj` in NVFP4; gated attention, Gated DeltaNet paths, embeddings, LM head, vision tower and MTP head stay BF16
Size	30.1 GB (from 55.6 GB BF16)
Calibration	mlabonne/harmless_alpaca, 128 samples, sequence length 512
Context length	262,144 tokens native
Hardware	1x NVIDIA RTX PRO 6000 Blackwell 96 GB (SM120)
Measured throughput	44 tok/s decode (SGLang, TP1, modelopt_fp4)
Toolchain	NVIDIA ModelOpt 0.46.0, SGLang 0.5.19, transformers 5.15.0

Refusal behavior (measured, hard markers only)

Effective refusal rate on the same 1120 harmful prompts used for the BF16 master, served at temperature 0 with thinking disabled, counting explicit decline phrases in the first 25 words.

Table with columns: Eval set, Prompts, Base model, BF16 master, This export
Eval set	Prompts	Base model	BF16 master	This export
JailbreakBench	100	71	0	0
tulu-harmbench	320	232	0	0
HarmfulQA	300	207	0	0
LLM-LAT

Run it with SGLang (validated)

bash

python -m sglang.launch_server \  --model-path ressl/Qwen3.8-27B-uncensored-NVFP4 \  --served-model-name Qwen3.8-27B-uncensored \  --quantization modelopt_fp4 --tp 1 \  --context-length 262144 --mem-fraction-static 0.85 \  --trust-remote-code

Quality & limitations

Quantization-aware caveat: residual refusals above the BF16 numbers are quantization-induced; the BF16 master answers the same prompts.
MTP speculative decoding is not part of this export (ModelOpt drops the mtp.* tensors); the vision tower is preserved.
The model is a reasoning model by default; pass enable_thinking=False for short direct answers.

Provenance & reproducibility

BF16 master: abliterated Qwen/Qwen3.8-27B with norm-preserving biprojection over 128 residual-writing matrices (pipeline in the master repository). Export: qwen38_export_nvfp4.py, quant audit nvfp4_report.json. Validation: the served artifact was evaluated end-to-end with the house cross-eval before upload.

License & credits

Apache-2.0 (inherited from Qwen3.8-27B).

Abliteration, quantization and validation by Robert Ressl (Hugging Face - Website - LinkedIn).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

ressl

Model Tree

Base

ressl/Qwen3.8-27B-uncensored

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality