GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
DevelopingDad
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
DevelopingDad
Model Tree
orcarouter/Qwen3.8-27B-Uncensored
Input Modalities
Output Modalities
Supported Functionality
The weights were produced from the immutable parent revision
9878936be9458522b5aeed0e13476bb8426f57f0
with NVIDIA Model Optimizer (ModelOpt) using its Qwen3.5
dense mixed-precision PTQ recipe and MSE calibration.
| Property | Value |
|---|---|
| Architecture | Qwen3_5ForConditionalGeneration |
| Weight layout | ModelOpt MIXED_PRECISION |
| NVFP4 targets | 193 W4A16_NVFP4 layers, including MLP projections and lm_head |
| FP8 targets | 208 attention and large linear-attention projections |
| Context configuration | 262,144 tokens |
| Export | 3 safetensors shards, about 21 GB |
| Preserved families | vision tower, MTP head, convolution and selected hybrid-state tensors |
| Calibration | 256 text rows, sequence length 2,048, batch size 1, MSE scale selection |
This is a quantized representation of the same OrcaRouter parent model, not a newly trained model. It is not a pure all-NVFP4 checkpoint.
The export contains FP8-KV metadata, but the validated production recipe deliberately uses BF16 KV and BF16 Mamba state. The headline benchmark does not use FP8 KV.
Integrity manifests:
WEIGHTS_SHA256SUMS covers the three published safetensors shards.SOURCE_ARTIFACT_SHA256SUMS covers every unchanged file from the original
C-MSE export. It intentionally excludes this repository's rewritten model card and added documentation.The accompanying ModelOpt study produced both max-calibrated and MSE-calibrated exports. Their older,
narrow quality rubric tied. In the later production gate, the max-calibrated artifact reached 33.88 tok/s
but answered the arithmetic check incorrectly (3463 instead of 3563) and was rejected.
This MSE artifact passed that arithmetic check and the complete serving gate, so it was the only artifact promoted.
Download this target and the external draft:
bash
hf download DevelopingDad/Qwen3.8-27B-Uncensored-NVFP4-MSE-DGX-Spark \ --local-dir /path/to/target hf download maurienne-ai/Qwen3.8-27B-DFlash2-NVFP4-RTNcal \ --revision bd7a934213c47a9e7ef69eef36bb3325f47fd1f1 \ --local-dir /path/to/draftLaunch the exact validated SGLang image:
bash
docker run --rm --gpus all --network host --ipc host --shm-size 32g \ -e HF_HUB_OFFLINE=1 \ -e TRANSFORMERS_OFFLINE=1 \ -e SGLANG_OPT_MAMBA_SKIP_DECODE_LOCK=0 \ -v /path/to/target:/models/target:ro \ -v /path/to/draft:/models/draft:ro \ lmsysorg/sglang@sha256:00205b89f74691f76a0ffbd6846376d9323971930a5d59bf63a65dadc7d67927 \ python3 -m sglang.launch_server \ --model-path /models/target \ --served-model-name Qwen3.8-27B-Uncensored-NVFP4-MSE \ --host 127.0.0.1 \ --port 30138 \ --mem-fraction-static 0.80 \ --sleep-on-idle \ --attention-backend flashinfer \ --fp8-gemm-backend cutlass \ --chunked-prefill-size 8192 \ --disable-prefill-cuda-graph \ --kv-cache-dtype bfloat16 \ --mamba-ssm-dtype bfloat16 \ --mamba-full-memory-ratio 4.21 \ --mamba-radix-cache-strategy extra_buffer \ --max-mamba-cache-size 20 \ --max-running-requests 4 \ --context-length 262144 \ --speculative-algorithm DFLASH \ --speculative-draft-model-path /models/draft \ --speculative-draft-model-quantization modelopt_fp4 \ --speculative-num-draft-tokens 8 \ --reasoning-parser qwen3 \ --tool-call-parser qwen3_coder \ --sampling-defaults model \ --enable-metrics \ --enable-cache-reportSee SERVING.md for the validation checklist and rollback notes.
Hardware: one NVIDIA DGX Spark / GB10 with 128 GB unified memory.
Primary workload: concurrency 1, temperature 0, thinking disabled, fixed 128 output tokens, one discarded warm-up followed by seven measured requests.
| System | Median decode speed | Range |
|---|---|---|
| Baseline system: parent FP8 target + unquantized DFlash2 | 18.278 tok/s | recorded separately |
| Winning system: this mixed target + calibrated NVFP4 DFlash2 | 34.821 tok/s | 34.793-34.940 |
| End-to-end system improvement | +90.5% |
Median time to first token improved from 0.2984 s to 0.1923 s (-35.6%). A five-content suite (programming list, Python code, systems prose, worked math, and French prose) improved from a pooled median of 24.062 to 34.008 tok/s (+41.3%). Every content-class median improved by more than 25%.
This is a system-level DGX Spark result, not an intrinsic speed guarantee for the weight files. The baseline used the FP8 target plus an unquantized DFlash2 draft; the winning system uses this mixed target plus the calibrated NVFP4 DFlash2 draft. It is not a pure target-weight ablation.
Machine-readable speed and capability details are in benchmark-results.json.
The promoted configuration was also evaluated on the same four benchmark families and sample counts reported by the OrcaRouter parent card. All 1,200 requests ran on one DGX Spark with temperature 0, thinking disabled, a fixed sample seed, and concurrency 4 (the service's configured low-concurrency ceiling).
The endpoint retained the legacy served-model alias orcarouter/Qwen3.8-27B-Uncensored-FP8. Container-mount
inspection and fresh SHA-256 checks of all three target shards confirmed that /models/target was this
published C-MSE artifact; the API alias was not used as the model-identity proof.
| Benchmark | N | Qwen Base | OrcaRouter | Ours | vs Base | vs Parent |
|---|---|---|---|---|---|---|
| MMLU (all, 0-shot letter) | 300 | 84.3% | 84.7% | 86.7% (260/300) | +2.4 pp | +2.0 pp |
| MMLU-Pro (CoT) | 250 | 77.6% | 76.8% | 77.6% (194/250) |
The Qwen-base and OrcaRouter-parent columns are quoted from the
orcarouter/Qwen3.8-27B-Uncensored capability table.
That card does not publish its sample IDs, random seed, exact prompts, generation limits, or scorer. This
model's column is therefore a new pinned run of the same benchmark families and sample counts, not a
matched rerun of the hidden upstream protocol. The deltas are descriptive and should not be read as proof
that quantization or SGLang improved the model's intrinsic capability.
The run used the pinned SGLang/OpenAI Simple Evals generative MMLU prompt, five category-matched CoT examples
for MMLU-Pro, the community-standard eight-example GSM8K CoT prompt, and zero-shot Chinese letter selection
for CMMLU. The GSM8K prompt difference is especially important: the parent card only says CoT, so the large
GSM8K delta must not be attributed to the checkpoint or runtime.
Every response was retained and failures were scored as failures. Six MMLU responses and 30 MMLU-Pro
responses did not produce an extractable final answer; all were counted incorrect. GSM8K and CMMLU had 100%
answer extraction. The complete run had zero request errors. Dataset revisions, sample seed, answer-extraction
rules, parse counts, and per-category results are recorded in benchmark-results.json.
The promoted configuration passed the full gate before restart, after a cold restart, and on the persistent production port:
Additional exact retrieval checks passed at 120,003 and 199,999 prompt tokens. These are targeted serving checks, not a comprehensive benchmark, safety evaluation, or proof of full 262K reasoning parity.
This repository contains transformed weight and configuration files. DevelopingDad produced the ModelOpt mixed-precision export and the accompanying DGX Spark validation materials; the parent model training and abliteration were performed upstream.
Qwen/Qwen3.8-27B.orcarouter/Qwen3.8-27B-Uncensored.Qwen3.8-27B-DFlash2-NVFP4-RTNcal
draft used by the reported serving configuration.This independent community derivative is not affiliated with, endorsed by, or sponsored by Alibaba Cloud/Qwen, OrcaRouter/Continuum AI, NVIDIA, maurienne.ai, Inco AI, SGLang, or the DFlash authors. Their names and trademarks are used only for identification and attribution.
See ATTRIBUTION.md for consolidated provenance, license, and external-component notes.
Qwen/Qwen3.8-27Borcarouter/Qwen3.8-27B-Uncensored9878936be9458522b5aeed0e13476bb8426f57f0maurienne-ai/Qwen3.8-27B-DFlash2-NVFP4-RTNcal
at bd7a934213c47a9e7ef69eef36bb3325f47fd1f1If you use this weight artifact, please cite the artifact and the upstream Qwen model. If you use the reported speculative-serving configuration, please also cite DFlash 2, the original DFlash work, and NVIDIA ModelOpt.
bibtex
@misc{developingdad2026qwen38nvfp4mse, title = {{Qwen3.8-27B-Uncensored-NVFP4-MSE-DGX-Spark}}, author = {{DevelopingDad}}, year = {2026}, publisher = {Hugging Face}, url = {https://huggingface.co/DevelopingDad/Qwen3.8-27B-Uncensored-NVFP4-MSE-DGX-Spark}, note = {ModelOpt C-MSE mixed NVFP4/FP8 derivative with a validated SGLang DFlash2 recipe}} @misc{qwen38, title = {{Qwen3.8-Max}: A New Bar for Coding and Cowork}, url = {https://qwen.ai/blog?id=qwen3.8}, author = {{Qwen Team}}, month = {August}, year = {2026}} @misc{inco2026dflash2, title = {{DFlash 2: Keep Drafting Parallel}}, author = {{Inco AI}}, year = {2026}, month = {August}, url = {https://inco.ai/blog/dflash2/}} @inproceedings{chen2026dflash, title = {{DFlash: Block Diffusion for Flash Speculative Decoding}}, author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian}, booktitle = {International Conference on Machine Learning (ICML)}, year = {2026}} @misc{nvidia-modelopt, author = {{NVIDIA Corporation}}, title = {{NVIDIA Model Optimizer}}, howpublished = {\url{https://github.com/NVIDIA/Model-Optimizer}}, year = {2024--2026}, note = {GitHub repository}}Apache 2.0. The included LICENSE is byte-for-byte identical to the license distributed by both the Qwen
base and the pinned OrcaRouter parent. Neither upstream repository included a NOTICE file at the audited
revisions. Retain upstream attribution and comply with the terms of any separately downloaded draft model.
| +0.8 pp |
| GSM8K (CoT) | 150 | 90.0% | 88.7% | 98.0% (147/150) | +8.0 pp | +9.3 pp |
| CMMLU (0-shot, Chinese) | 500 | 81.4% | 80.8% | 81.8% (409/500) | +0.4 pp | +1.0 pp |
incoai/Qwen3.8-27B-DFlash2