Jiunsong

SuperQwen3.8-abliterated-100-fp8

README

License: apache-2.0

What this release is

SuperQwen3.8-abliterated-100-fp8 is a directly loadable Qwen3.8-27B derivative for teams that need fewer blanket refusals in legitimate research, security, policy, healthcare, retail, compliance, and creative workflows. It keeps the upstream multimodal, reasoning, long-context, and tool-use paths while applying a wider OBLITERATUS-derived refusal-subspace projection than the earlier SuperQwen3.8 release.

The public artifact is already quantized as FP8_DYNAMIC W8A8 using compressed-tensors: per-channel FP8 weights and dynamic per-token FP8 activations. That is the format intended for H100/H200 Tensor Cores; no calibration dataset is required at load time.

Release highlights

Table with columns: Verified release value
	Verified release value
Parent	`Qwen/Qwen3.8-27B` at `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`
Refusal directions	842 canonical paired prompts, per-layer rank 8
Projection	Language layers 12–63, embeddings 0.65 blend, output head 1.0 blend, norm-preserving
FP8 format	FP8_DYNAMIC W8A8, per-channel weights + dynamic per-token activations
Protected paths	Vision 333 tensors, conv1d 48, MTP 15, and `lm_head` retained exactly outside FP8 packing
Checkpoint	7 FP8 target shards + protected BF16 MTP sidecar, about 29.08 GiB
Runtime kernel	vLLM selected `CutlassFP8ScaledMMLinearKernel` for `CompressedTensorsW8A8Fp8`
Capability	8/8 deterministic checks
Tool use	OpenAI function call parsed with exact name and arguments
Benign-sensitive gate	0/8 explicit refusals
OBLITERATUS audit	0/64 refusals, 0 blank outputs, 0 token leaks or loops
Vision	PASS
Bounded reasoning	default/low/medium PASS; xhigh completed correctly at 240 reasoning tokens
Speculative decoding	Native MTP K=5; 22.6322 tok/s at p256 and 21.4891 tok/s after an 8K prompt on GB10
Long context	65,579 actual prompt tokens retrieved on the final 262,144-token K=5 server
Measured hardware	1× NVIDIA DGX Spark / GB10, C1, TP=1 — not an H100/H200 throughput claim

Stronger refusal reduction

This release was rebuilt from the pinned parent rather than patched from an already modified checkpoint. OBLITERATUS revision a5a1ffa5849b442cf188b3c03fd4de71ddf5bdcc supplied 842 canonical harmful/harmless pairs. A per-layer rank-8 subspace was fit and projected through the residual output paths of layers 12–63, plus bounded projections of the embeddings and output head.

The independent geometric reload gate passed all 106 modified tensors. Typical post-projection component residuals were about 0.12–0.17%, below the fixed 2% ceiling. The direction artifact SHA-256 is f2a0ba2b3c9e3faad158ac475732f48c84561c6f7a512cfd6a0e69558d64fbe1 and the independently reloaded BF16 tensor-manifest identity is 1679546a7769831a90f303d5389b168d2218a04006b6e09bc2c6e9bd7f57a2b2.

“Abliterated” means the measured refusal direction was substantially reduced. It does not mean every possible refusal has disappeared, and it does not turn generated text into verified business, legal, medical, or security advice.

FP8 checkpoint

Table with columns: Component, Precision / treatment
Component	Precision / treatment
Eligible language-backbone linear weights	FP8, per-channel
Input activations	FP8, dynamic per-token
Vision tower	BF16, exact
Conv1d / hybrid-state paths	BF16, exact
Native MTP draft head	BF16 sidecar, exact
`lm_head`	BF16, exact
Recommended KV cache	FP8

Structural verification found 496 language-backbone FP8 scale tensors, no quantization sidecars on protected modules, and exact equality for every protected tensor. The full repository is covered by SHA256SUMS.json.

H100 / H200 serving

This checkpoint is sized for TP=1 on one H100 80GB or H200 141GB. Use TP=2 when your workload values prefill concurrency or operational headroom more than single-GPU latency. The commands below are deployment profiles, not fabricated Hopper benchmarks.

One GPU, production baseline

bash
vllm serve Jiunsong/SuperQwen3.8-abliterated-100-fp8 \
  --served-model-name SuperQwen3.8-abliterated-100-fp8 \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.92 \
  --kv-cache-dtype fp8 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --async-scheduling \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3

Two GPUs

bash
vllm serve Jiunsong/SuperQwen3.8-abliterated-100-fp8 \
  --served-model-name SuperQwen3.8-abliterated-100-fp8 \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.92 \
  --kv-cache-dtype fp8 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --async-scheduling \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3

For H100/H200, keep the native compressed-tensors FP8 path. Converting this release back to BF16 before serving discards the point of the Hopper build. Benchmark your own prompt lengths and concurrency before fixing production limits.

Speculative decoding

The repository retains the exact native Qwen MTP head. K=5 is the selected fast profile. It passed the complete capability, tool, benign-sensitive, bounded-reasoning, vision, 8K throughput, and 32,811-token retrieval gates. On the measured GB10 runtime:

Table with columns: Profile, p256 C1 decode, 8K-prompt C1 decode, Result
Profile	p256 C1 decode	8K-prompt C1 decode	Result
K=0	7.8140 tok/s	7.7224 tok/s	Stable baseline
K=3 + prefix cache	18.9919 tok/s	—	Rejected: 8K run stalled
K=5, prefix cache off	22.6322 tok/s	21.4891 tok/s	Selected

K=5 is about 2.90× the measured short-prompt K=0 decode rate. Across the final K=5 validation process the server accepted 1,587 of 3,135 drafted token positions; acceptance varied by workload. The bundled launcher therefore defaults to K=5 and does not combine speculation with prefix caching. Set MTP_TOKENS=0 for the most conservative path.

Measured GB10 baseline

The reproducible non-speculative benchmark follows the post-first-token decode window used by MiaAI-Lab/sparkDash at commit bf2709a80ef25d0e1a6ee41efec4c9b8042a5b8b.

Table with columns: Prompt class, Concurrency, Decode, TTFT
Prompt class	Concurrency	Decode	TTFT
256 target tokens (292 after chat formatting)	C1	7.8140 tok/s	0.261 s
8,192 target tokens (8,222 after chat formatting)	C1	7.7224 tok/s	10.801 s

These are DGX Spark / GB10 measurements in eager mode. They are not estimates for H100 or H200. Hopper owners should expect different results and should publish the exact GPU, vLLM revision, prompt tokens, output tokens, concurrency, and decode window when comparing deployments.

Release gates

The release gate stores prompt and output hashes rather than redistributing raw test content.

Table with columns: Gate, Result
Gate	Result
FP8 structure + exact protected tensors	PASS
Independent reload of projected BF16 source	PASS
Deterministic capability	8/8
Tool call	PASS
Benign-sensitive refusal	0/8
Vision	PASS
Bounded reasoning	PASS through xhigh
32K retrieval, K=0 and K=5	PASS / PASS
65K retrieval on final 262,144-token K=5 server

API example

python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
response = client.chat.completions.create(
    model="SuperQwen3.8-abliterated-100-fp8",
    messages=[{"role": "user", "content": "Draft a concise supplier risk memo."}],
    temperature=0.2,
)
print(response.choices[0].message.content)

For bounded reasoning, pass chat_template_kwargs through your vLLM client:

python
extra_body={"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "medium"}}

Limitations

Abliteration reduces measured refusal behavior; it does not guarantee universal compliance, factuality, safety, or suitability for a particular business decision.
FP8 can regress workloads outside the measured suites. Validate your domain data.
Tool calls must be authorized, sandboxed, logged, and checked by the application.
Long-context capacity is not the same as perfect long-context recall.
Speed varies with GPU, driver, vLLM build, prompt length, output length, batching, multimodal inputs, and sampling settings.
H100/H200 commands are optimized launch guidance; only GB10 numbers are presented as measurements in this card.

Evidence identities

The final public artifact includes hash-only release reports, quantization verification, the exact build recipe, and a file-by-file SHA-256 manifest. Large files are uploaded as Git LFS objects and their remote LFS OIDs are verified against the local SHA-256 values before the repository is made public.

License

Apache-2.0, following the upstream Qwen3.8 release.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Jiunsong

Model Tree

Base

Qwen/Qwen3.8-27B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality