GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
AudioJaguar
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
AudioJaguar
Model Tree
orcarouter/Qwen3.8-27B-Uncensored
Input Modalities
Output Modalities
Supported Functionality
DFlash2 drafts a block of tokens per step and the target verifies them. That makes decode
even more bandwidth-bound than usual, and it adds one specific cost: the drafter's
candidate selector reads the target's lm_head a second time every step, on top of
the verify pass. A BF16 head is 2.5 GB; NVFP4 is 0.7 GB. Paid twice per step, that
difference is worth real throughput on a memory-bandwidth-limited card.
So everything that can be NVFP4 is NVFP4 — all 64 layers' MLP projections and the output head — with FP8 reserved for the attention and linear-attention projections.
| Modules | Format |
|---|---|
All 64 layers' MLP gate_proj / up_proj / down_proj, plus lm_head | NVFP4 W4A4, group 16 (193 modules) |
Self-attention q/k/v/o_proj; Gated DeltaNet in_proj_qkv / in_proj_z / out_proj | FP8 W8A8 (208 modules) |
| KV cache | FP8 scales |
Vision tower, MTP head, conv1d, in_proj_a/b, embeddings, norms | BF16 |
hf_quant_config.json's per-layer map is byte-identical to RadixArk's pinned revision.
The MTP head (15 tensors) is preserved unquantized and listed in exclude_modules, so
MTP speculative decoding remains available as a fallback if you'd rather not run DFlash2.
Pairs with the z-lab/Qwen3.8-27B-DFlash2
drafter (BF16, ~3.85 GB). SGLang flags:
markdown
--speculative-algorithm DFLASH \--speculative-draft-model-path z-lab/Qwen3.8-27B-DFlash2 \--speculative-num-draft-tokens 8 \--speculative-draft-model-quantization unquant
Keep the drafter unquantized — a quantized drafter loses more in acceptance than it saves in bandwidth, and quantized-drafter support is still incomplete upstream.
DFlash2 support is recent. At time of writing it requires a build carrying SGLang PR
#35371 plus a fix allowing the candidate selector to run through a quantized lm_head
— which this checkpoint has. The failure mode when an engine can't is usually a crash or
silently disabled speculation, not wrong output. Speculative decoding is lossless: the
target verifies every drafted token, so acceptance affects speed only, never output.
lm_head trade-offA quantized lm_head shifts the candidate selector's top-k slightly, which can lower
draft acceptance. RadixArk moved their own checkpoint's head back to BF16 for that reason
— a sensible call on a GB300, where 3.4 GB/step is noise. On a GB10 that same 3.4 GB is
about 15% of the step, and the NVFP4-head layout came out faster end-to-end despite any
acceptance cost. Which side you land on depends on your memory bandwidth. If you want a
BF16-head variant, the recipe in this repo has a comment showing the two-line change.
modelopt-commit.txtorca-nvfp4-w4a4-fp8attn.yaml (in this repo)cnn_dailymail 3.0.0, 1024 samples, seq len 512, batch 8, max-calibrationquant_summary.txtApache 2.0, inherited from Qwen/Qwen3.8-27B via orcarouter/Qwen3.8-27B-Uncensored. Layout follows RadixArk/Qwen3.8-27B-NVFP4. Drafter by z-lab.