0xWhiteMage

Qwen3.8-27B-Kearuga-NVFP4

README

License: apache-2.0

📌 Model Overview

Qwen3.8-27B-Kearuga-NVFP4 is a high-performance, tiered-sensitivity quantized checkpoint of Qwen3.8-27B engineered specifically for high-throughput inference and speculative decoding with SGLang on the 128 GB NVIDIA DGX Spark.

Unlike uniform 4-bit quantizations that suffer from logit tail collapse, Kearuga applies an EXL3-inspired sensitivity hierarchy:

Tier 1 (BF16): embed_tokens, lm_head, and boundary layers 0–1 & 62–63 are preserved in high-precision to guarantee exact vocabulary framing and syntax fidelity.
Tier 2 (FP8 E4M3): Attention projections ( $Q, K, V, O$ ) and Gated DeltaNet linear recurrence ( $in_p r o j$ ) are quantized to FP8 to eliminate recurrent state drift over 262K contexts and protect speculative draft feature taps.
Tier 3 (NVFP4): Middle MLP blocks (layers 2–61) are quantized to hardware-native NVFP4 for maximum Blackwell Tensor Core compute throughput.

📊 Benchmark & Mathematical Fidelity

Table with columns: Benchmark / Metric, Original Qwen3.8-27B (BF16), Uniform NVFP4 Baseline, 🧙‍♂️ Qwen3.8-27B-Kearuga-NVFP4
Benchmark / Metric	Original Qwen3.8-27B (BF16)	Uniform NVFP4 Baseline	🧙‍♂️ Qwen3.8-27B-Kearuga-NVFP4
Weight Footprint	54.2 GiB	~23.0 GiB	31.37 GiB (-42.1%)
KL Divergence ( $D KL $ )	0.000 (Reference)	0.112 (Tail noise)	0.038 (Lossless parity)

🚀 Quickstart with SGLang

Pair with the official distilled drafter 0xWhiteMage/Qwen3.8-27B-Kearuga-DFlash2-FP8-E4M3 for up to 65–82 tok/s single-stream decode speed:

bash
python3 -m sglang.launch_server \
  --model-path 0xWhiteMage/Qwen3.8-27B-Kearuga-NVFP4 \
  --speculative-draft 0xWhiteMage/Qwen3.8-27B-Kearuga-DFlash2-FP8-E4M3 \
  --speculative-algorithm DFLASH \
  --speculative-num-draft-tokens 8 \
  --kv-cache-dtype fp8_e4m3 \
  --context-length 262144 \
  --max-total-tokens 1048576 \
  --cuda-graph-max-bs-decode 4 \
  --host 0.0.0.0 --port 8888

🤝 Community & GitHub

For full launch scripts, Docker recipes, and kernel overlays: 👉 GitHub: 0xWhiteMage/Qwen3.8-27B-Kearuga-SGLang-DGX-Spark-DFlash2

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

0xWhiteMage

Model Tree

Base

Qwen/Qwen3.8-27B

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality