0xWhiteMage

Qwen3.8-27B-Kearuga

README

License: apache-2.0

⚡ Overview

Qwen3.8 27B Kearuga is a custom tiered sensitivity hybrid-quantized checkpoint of Qwen/Qwen3.8-27B, specifically engineered for high-throughput and interactive speculative-decoding serving on a single NVIDIA DGX Spark (GB10 / Grace-Blackwell, 128 GB Unified Memory).

By pairing GPTQ with Four-Over-Six (4o6) group scales, NVFP4 AWQ, FP8 E4M3, and protected BF16, Kearuga:

Cuts weight footprint by 52% (from 51.8 GiB unquantized base down to 24.85 GB across 3 shards + MTP draft head).
Preserves output fidelity to within a 0.0165 mean KL divergence against the unquantized BF16 base model, maintaining 40/40 (100%) top-1 argmax agreement.
Unlocks ultra-fast speculative serving with SGLang: Delivers 57 tok/s (C1, TTFT 264ms), 51 tok/s (C2 agg), and 94 tok/s (C4 agg, TTFT 480ms) interactive decode with stock DFlash 2.
Runs on official SGLang Docker images with zero kernel overlays or custom compilation required.

📊 Kearuga vs. Base Qwen3.8-27B (BF16)

The table below presents verified empirical metrics measured on an isolated NVIDIA DGX Spark (GB10 / SM121, 128 GB Unified Memory) running SGLang with DFlash 2 (K=10 draft block, KV cache in BF16, thinking disabled, temperature 0):

Table with columns: Metric, Base Model: Qwen/Qwen3.8-27B (Native BF16), Qwen3.8 27B Kearuga (Hybrid Quantized), Delta / Operational Significance
Metric	Base Model: `Qwen/Qwen3.8-27B` (Native BF16)	Qwen3.8 27B Kearuga (Hybrid Quantized)	Delta / Operational Significance
Weight Footprint	51.8 GiB	24.85 GB	−52.0% (fits with 64+ GiB headroom for KV cache)
Fidelity-40 Mean KL Divergence	0.0000	0.0165	−50.6% KL reduction vs legacy NVFP4 baseline (0.0334)
Fidelity-40 Mean Jensen-Shannon (JS)	0.0000	0.0034	Virtually identical distribution profile

🛡️ Tiered Sensitivity Quantization Architecture

Uniform quantization (applying INT4 or NVFP4 indiscriminately across all layers) degrades attention sink states, vocabulary distribution tails, and speculative draft features.

Inspired by empirical mixed-precision sensitivity research (EXL3 tiered sensitivity), Kearuga partitions the model's 2,009 tensors into four distinct sensitivity tiers:

Table with columns: Tier, Module Scope, Precision, Count, Architectural Purpose
Tier	Module Scope	Precision	Count	Architectural Purpose
Tier 1 (Protect)	`embed_tokens`, `lm_head`, all RMSNorms, visual tower (27 blocks / 333 weights), and MTP head tensors	Native BF16	348	Protects vocabulary tail, multi-modal reasoning, and MTP draft survival
Tier 2 (Medium)	Self-Attention (`q_proj`, `k_proj`, `v_proj`, `o_proj`), GDN Recurrence () (Layers 2–61), and Boundary MLPs (Layers 0, 1, 62, 63)

1. Four-Over-Six (4o6) Group Scaling

Standard GPTQ assigns a single static exponent code per group block (amax → code 6). Kearuga's 4o6 algorithm evaluates dynamic range block-by-block and selects the optimal scale:

Code 6: Standard headroom for high-variance weights.
Code 4: Higher numerical precision for weights with tight local distributions.

In our calibration, 44.7% of blocks selected Code 4, yielding a 16% reduction in Hessian-weighted MSE (0.529 → 0.445) and cutting held-out KL divergence by 21.7% without increasing tensor bit-width or memory footprint.

2. Fused-Shared Global Scale Contract

SGLang's high-performance Blackwell kernels fuse gate_proj and up_proj into a single GEMM operation, indexing a unified weight_scale_2.max(). Quantizing gate and up independently introduces scale divergence that breaks dequantization. Kearuga enforces an exact fused-shared global scale across every layer:

text
scale_shared = max(amax(gate), amax(up))

This ensures bit-exact kernel compatibility and eliminates dequantization distortion.

🚀 Serving & Deployment Guide

Kearuga is served directly via official SGLang Docker container images (sha256:616a3e97… or later) with native DFlash 2 speculative acceleration. The speculative drafter uses the stock drafter z-lab/Qwen3.8-27B-DFlash2:

bash
docker run --gpus all \
  --shm-size 32g \
  --ulimit memlock=-1:-1 \
  --cap-add IPC_LOCK \
  -p 8888:8888 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe \
  python3 -m sglang.launch_server \
    --model-path 0xWhiteMage/Qwen3.8-27B-Kearuga \
    --served-model-name "Qwen3.8 27B Kearuga" \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path z-lab/Qwen3.8-27B-DFlash2 \
    --speculative-num-draft-tokens 10 \
    --speculative-draft-window-size 2048 \
    --kv-cache-dtype bf16 \
    --mem-fraction-static 0.78 \
    --context-length 262144 \
    --max-running-requests 4 \
    --host 0.0.0.0 \
    --port 8888

Key Serving Parameters

Table with columns: Parameter, Recommended Setting, Rationale
Parameter	Recommended Setting	Rationale
`--speculative-num-draft-tokens`	`10`	Validated in empirical sweeps: K=8 drops code/math acceptance by 7–12%; K=12/16 degrade prose/IFEval and C4 throughput. K=10 represents the Pareto-optimal draft length.
`--kv-cache-dtype`	`bf16`	Preserves full logit fidelity (KL: 0.0170 → 0.0165, exact continuations: 20/40 vs 19/40) with negligible compute cost.
`--context-length`	`262144`	Full native context supported without YaRN interpolation.

⏱️ Saturated Priority Scheduling

In multi-agent production pipelines, background batch traffic can cause interactive request freezes. Kearuga's serving stack enables native priority preemption:

json
{
  "model": "Qwen3.8 27B Kearuga",
  "priority": 100,
  "messages": [
    {"role": "user", "content": "Analyze this mission-critical trace..."}
  ]
}

Table with columns: Server Load State, Default Priority TTFT, Interactive Priority ("priority": 100), Latency Reduction
Server Load State	Default Priority TTFT	Interactive Priority (`"priority": 100`)	Latency Reduction
DFlash 2 (All 4 Seats Saturated)	~43.15 s	~2.63 s	93.9% faster

🧪 Comprehensive Verification & Quality Gates

Every promoted build of Kearuga must satisfy a strict 15-gate fail-closed verification harness prior to promotion:

Static Tensor Census: 2,009 tensors intact across 3 shards; exactly 15 MTP tensors and 333 visual tensors confirmed byte-identical to base BF16.
Serving Contract Audit: Strict verification that fused gate_proj + up_proj weight_scale_2 are identical across all layers.
Semantic Canaries: 100% pass on exact arithmetic (19 × 23 = 437) and logic traps (9.9 vs 9.11).
Long-Context NIAH: 3/3 Needle-In-A-Haystack retrieval at 64K depth (25%, 50%, 90%).
Quality-200 Benchmark: 157/180 verified score on GSM8K, HumanEval, IFEval, and agentic coding.
Multi-Boot Stability: 3 consecutive clean cold boots with zero loader warnings or device-side asserts.

📦 Model Files & Shards

Table with columns: File Name, File Size, Description & Scope
File Name	File Size	Description & Scope
`config.json`	8.2 KB	Architectural model hyperparameters & quantization configuration
`generation_config.json`	240 B	Sampling defaults, EOS tokens, and generation bounds
`hf_quant_config.json`	14.1 KB	Detailed ModelOpt mixed-precision tier mappings & scale rules
`model.safetensors.index.json`	185 KB

DFlash 2 Speculative Drafter: z-lab/Qwen3.8-27B-DFlash2
Base Model (BF16): Qwen/Qwen3.8-27B
Creator / Maintainer: The White Mage · @0xWhiteMage on X

📜 Citation

If you use Kearuga or build upon its tiered sensitivity architecture in your work, please cite:

bibtex
@misc{kearuga-2026,
  title={Kearuga: Hybrid GPTQ-4o6 + FP8 Quantization of Qwen3.8-27B for Speculative-Decoding Serve on NVIDIA DGX Spark},
  author={0xWhiteMage},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/0xWhiteMage/Qwen3.8-27B-Kearuga}
}

📄 License

This checkpoint is released under the Apache 2.0 License. The underlying base model Qwen/Qwen3.8-27B is subject to its original license terms from Alibaba Cloud.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

0xWhiteMage

Model Tree

Base

Qwen/Qwen3.8-27B

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

⚡ Overview

By pairing GPTQ with Four-Over-Six (4o6) group scales, NVFP4 AWQ, FP8 E4M3, and protected BF16, Kearuga:

Cuts weight footprint by 52% (from 51.8 GiB unquantized base down to 24.85 GB across 3 shards + MTP draft head).
Preserves output fidelity to within a 0.0165 mean KL divergence against the unquantized BF16 base model, maintaining 40/40 (100%) top-1 argmax agreement.
Unlocks ultra-fast speculative serving with SGLang: Delivers 57 tok/s (C1, TTFT 264ms), 51 tok/s (C2 agg), and 94 tok/s (C4 agg, TTFT 480ms) interactive decode with stock DFlash 2.
Runs on official SGLang Docker images with zero kernel overlays or custom compilation required.

📊 Kearuga vs. Base Qwen3.8-27B (BF16)

Table with columns: Metric, Base Model: Qwen/Qwen3.8-27B (Native BF16), Qwen3.8 27B Kearuga (Hybrid Quantized), Delta / Operational Significance
Metric	Base Model: `Qwen/Qwen3.8-27B` (Native BF16)	Qwen3.8 27B Kearuga (Hybrid Quantized)	Delta / Operational Significance
Weight Footprint	51.8 GiB	24.85 GB	−52.0% (fits with 64+ GiB headroom for KV cache)
Fidelity-40 Mean KL Divergence	0.0000	0.0165	−50.6% KL reduction vs legacy NVFP4 baseline (0.0334)
Fidelity-40 Mean Jensen-Shannon (JS)	0.0000	0.0034	Virtually identical distribution profile

🛡️ Tiered Sensitivity Quantization Architecture

Uniform quantization (applying INT4 or NVFP4 indiscriminately across all layers) degrades attention sink states, vocabulary distribution tails, and speculative draft features.

Inspired by empirical mixed-precision sensitivity research (EXL3 tiered sensitivity), Kearuga partitions the model's 2,009 tensors into four distinct sensitivity tiers:

Table with columns: Tier, Module Scope, Precision, Count, Architectural Purpose
Tier	Module Scope	Precision	Count	Architectural Purpose
Tier 1 (Protect)	`embed_tokens`, `lm_head`, all RMSNorms, visual tower (27 blocks / 333 weights), and MTP head tensors	Native BF16	348	Protects vocabulary tail, multi-modal reasoning, and MTP draft survival
Tier 2 (Medium)	Self-Attention (`q_proj`, `k_proj`, `v_proj`, `o_proj`), GDN Recurrence () (Layers 2–61), and Boundary MLPs (Layers 0, 1, 62, 63)

1. Four-Over-Six (4o6) Group Scaling

Standard GPTQ assigns a single static exponent code per group block (amax → code 6). Kearuga's 4o6 algorithm evaluates dynamic range block-by-block and selects the optimal scale:

Code 6: Standard headroom for high-variance weights.
Code 4: Higher numerical precision for weights with tight local distributions.

2. Fused-Shared Global Scale Contract

text
scale_shared = max(amax(gate), amax(up))

This ensures bit-exact kernel compatibility and eliminates dequantization distortion.

🚀 Serving & Deployment Guide

bash
docker run --gpus all \
  --shm-size 32g \
  --ulimit memlock=-1:-1 \
  --cap-add IPC_LOCK \
  -p 8888:8888 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe \
  python3 -m sglang.launch_server \
    --model-path 0xWhiteMage/Qwen3.8-27B-Kearuga \
    --served-model-name "Qwen3.8 27B Kearuga" \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path z-lab/Qwen3.8-27B-DFlash2 \
    --speculative-num-draft-tokens 10 \
    --speculative-draft-window-size 2048 \
    --kv-cache-dtype bf16 \
    --mem-fraction-static 0.78 \
    --context-length 262144 \
    --max-running-requests 4 \
    --host 0.0.0.0 \
    --port 8888

Key Serving Parameters

Table with columns: Parameter, Recommended Setting, Rationale
Parameter	Recommended Setting	Rationale
`--speculative-num-draft-tokens`	`10`	Validated in empirical sweeps: K=8 drops code/math acceptance by 7–12%; K=12/16 degrade prose/IFEval and C4 throughput. K=10 represents the Pareto-optimal draft length.
`--kv-cache-dtype`	`bf16`	Preserves full logit fidelity (KL: 0.0170 → 0.0165, exact continuations: 20/40 vs 19/40) with negligible compute cost.
`--context-length`	`262144`	Full native context supported without YaRN interpolation.

⏱️ Saturated Priority Scheduling

In multi-agent production pipelines, background batch traffic can cause interactive request freezes. Kearuga's serving stack enables native priority preemption:

json
{
  "model": "Qwen3.8 27B Kearuga",
  "priority": 100,
  "messages": [
    {"role": "user", "content": "Analyze this mission-critical trace..."}
  ]
}

Table with columns: Server Load State, Default Priority TTFT, Interactive Priority ("priority": 100), Latency Reduction
Server Load State	Default Priority TTFT	Interactive Priority (`"priority": 100`)	Latency Reduction
DFlash 2 (All 4 Seats Saturated)	~43.15 s	~2.63 s	93.9% faster

🧪 Comprehensive Verification & Quality Gates

Every promoted build of Kearuga must satisfy a strict 15-gate fail-closed verification harness prior to promotion:

Static Tensor Census: 2,009 tensors intact across 3 shards; exactly 15 MTP tensors and 333 visual tensors confirmed byte-identical to base BF16.
Serving Contract Audit: Strict verification that fused gate_proj + up_proj weight_scale_2 are identical across all layers.
Semantic Canaries: 100% pass on exact arithmetic (19 × 23 = 437) and logic traps (9.9 vs 9.11).
Long-Context NIAH: 3/3 Needle-In-A-Haystack retrieval at 64K depth (25%, 50%, 90%).
Quality-200 Benchmark: 157/180 verified score on GSM8K, HumanEval, IFEval, and agentic coding.
Multi-Boot Stability: 3 consecutive clean cold boots with zero loader warnings or device-side asserts.

📦 Model Files & Shards

Table with columns: File Name, File Size, Description & Scope
File Name	File Size	Description & Scope
`config.json`	8.2 KB	Architectural model hyperparameters & quantization configuration
`generation_config.json`	240 B	Sampling defaults, EOS tokens, and generation bounds
`hf_quant_config.json`	14.1 KB	Detailed ModelOpt mixed-precision tier mappings & scale rules
`model.safetensors.index.json`	185 KB

DFlash 2 Speculative Drafter: z-lab/Qwen3.8-27B-DFlash2
Base Model (BF16): Qwen/Qwen3.8-27B
Creator / Maintainer: The White Mage · @0xWhiteMage on X

📜 Citation

If you use Kearuga or build upon its tiered sensitivity architecture in your work, please cite:

bibtex
@misc{kearuga-2026,
  title={Kearuga: Hybrid GPTQ-4o6 + FP8 Quantization of Qwen3.8-27B for Speculative-Decoding Serve on NVIDIA DGX Spark},
  author={0xWhiteMage},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/0xWhiteMage/Qwen3.8-27B-Kearuga}
}

📄 License

This checkpoint is released under the Apache 2.0 License. The underlying base model Qwen/Qwen3.8-27B is subject to its original license terms from Alibaba Cloud.

Qwen3.8-27B-Kearuga

README

⚡ Overview

📊 Kearuga vs. Base Qwen3.8-27B (BF16)

🛡️ Tiered Sensitivity Quantization Architecture

1. Four-Over-Six (4o6) Group Scaling

2. Fused-Shared Global Scale Contract

🚀 Serving & Deployment Guide

Key Serving Parameters

⏱️ Saturated Priority Scheduling

🧪 Comprehensive Verification & Quality Gates

📦 Model Files & Shards

🔗 Related Resources

📜 Citation

📄 License

Explore FriendliAI today

README

⚡ Overview

📊 Kearuga vs. Base Qwen3.8-27B (BF16)

🛡️ Tiered Sensitivity Quantization Architecture

1. Four-Over-Six (4o6) Group Scaling

2. Fused-Shared Global Scale Contract

🚀 Serving & Deployment Guide

Key Serving Parameters

⏱️ Saturated Priority Scheduling

🧪 Comprehensive Verification & Quality Gates

📦 Model Files & Shards

🔗 Related Resources

📜 Citation

📄 License