GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
PassingByPixels
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
PassingByPixels
Model Tree
nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451
Input Modalities
Output Modalities
Supported Functionality
The upstream bf16 repo was re-uploaded on 2026-07-21 to add
mtp.fc.weight - the fusion projection the MTP head needs to combine its two
inputs. A checkpoint pulled before that date has mtp.layers.0.* but no
mtp.fc, and this is a silent failure.
vLLM disables strict weight-initialisation checking for quantized configs. The missing tensor is allocated, nothing is loaded into it, and it keeps whatever was in that memory. The MTP head then drafts from a random projection: no exception, no warning, nothing in the log. Every drafted token is rejected, you pay the drafting cost for nothing, and decode gets slower. The only symptom is a number you were probably not measuring.
Check your own copy:
python
import json, urllib.requestidx = json.load(urllib.request.urlopen("https://huggingface.co/<repo>/resolve/main/model.safetensors.index.json"))print("mtp.fc.weight:", "mtp.fc.weight" in idx["weight_map"])
False means MTP will not work for you, however healthy it looks.
Tested on vLLM 0.25.2.dev0 (arm64/CUDA 13 build), DGX Spark GB10, sm_121a.
Plain, no speculative decoding:
bash
vllm serve PassingByPixels/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-NVFP4-MTP \--kv-cache-dtype fp8 --attention-backend flashinfer \--max-model-len 131072 --gpu-memory-utilization 0.72 --max-num-seqs 64 \--reasoning-parser qwen3 --trust-remote-code \--enable-auto-tool-choice --tool-call-parser qwen3_xml
With MTP (recommended): add
bash
--speculative-config '{"method":"mtp","num_speculative_tokens":2}'
No separate draft model - vLLM rewrites this checkpoint's own config into a
Qwen3_5MTP draft and reads the head out of these weights. It is a reasoning
model, so pass chat_template_kwargs: {"enable_thinking": false} if you do not
want the budget spent inside <think>.
With DFlash (much faster on file-editing work, much slower on prose, and it needs a patched draft because the published one does not load on vLLM at all): see the recipe repo linked below.
540 measurements on this checkpoint: 3 configs x 3 workloads x 10 concurrency levels x 6 shuffled rounds, all clean, standard deviation mostly under 1%.
An AI writes one word at a time, and each word means re-reading the whole model from memory. That memory trip is the slow part. Speculative decoding adds a small fast helper that guesses the next few words so the big model can check them in one go. Right guesses are free. Wrong guesses are thrown away and cost you time.
So it is a bet on whether the next words are predictable.
Text is predictable when the model is mostly copying something you just gave it - "here is my file, rename this variable" - which is what coding assistants do all day. It is unpredictable when the model is writing something new.
How often each helper guessed right, on this model:
| writing prose | writing code | editing a file | |
|---|---|---|---|
| MTP | 47% | 76% | 100% |
| DFlash | 6% | 19% | 83% |
DFlash guessing right 6% of the time on prose is the whole reason it is not a magic speed-up button.
Rank the workloads by that hit rate and you have ranked them by outcome, for both methods, at every concurrency level. It is not really about concurrency.
Aggregate tokens/sec vs no speculative decoding:
| workload | MTP vs no-spec |
|---|---|
| code | wins at every concurrency level tested, up to +70% |
| edit | wins at every level, up to +104% |
| prose | wins at 1-8, 24, 32 streams (up to +32%); loses at 12 (-11%), 16 (-10%), 64 (-7%) |
It costs about 10% of the KV cache pool and nothing else. On code and editing there is no concurrency at which it is the wrong choice. On free-form prose above 8 concurrent streams it is roughly a coin flip - that is where its acceptance is weakest (47%).
Writing code - MTP (orange) leads base (blue) at every level:
Editing files - DFlash (green) is in another league, MTP still well clear of base:
Free-form prose - DFlash drops below doing nothing from 4 streams up:
Single stream on editing work it is +716% over no speculation and +311% over MTP. That is real. But:
Ask for 64 streams and count how many actually run at once. Base and MTP track the diagonal; DFlash flattens at 22, so its throughput lines beyond that point are a capacity limit rather than a speed result:
Rule of thumb: DFlash below its ~22-stream ceiling for edit-heavy agentic coding; MTP for everything else and for crowds.
Full charts, tables, the 540-cell dataset and the fix that makes DFlash load on vLLM at all:
-> Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-NVFP4_Dflash_DGX_recipe
Averaging prose/code/edit assumes editing is exactly a third of your traffic. 72% of DFlash's average comes from the edit workload alone. Drop editing and average only prose and code, and MTP wins from 4 streams up. Measure your own mix instead - acceptance rate tells you directly:
bash
curl -s http://127.0.0.1:8000/metrics \| grep -E 'spec_decode_num_(accepted|draft)_tokens_total'
Near 0.06 is prose-like (do not use DFlash); near 0.83 is edit-like (do).
Source: nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451 - bf16, 1199
tensors, 55,457,998,304 bytes, a nuslerp mergekit merge of two Qwen3.6-27B
Architect-Polaris Fable variants. Uploaded with nightmedia's permission.
Quantised with NVIDIA ModelOpt 0.45.0, examples/llm_ptq/hf_ptq.py:
markdown
--qformat nvfp4 --calib_size 512 --dataset cnn_dailymail--batch_size 0 --inference_tensor_parallel 1
ModelOpt auto-excludes linear_attn, the vision tower, mtp.*, embeddings and
lm_head from quantisation - see quant_summary.txt in this repo for the
per-layer record, and hf_quant_config.json for the exclusion patterns.
Output: 2398 tensors, 20,487,991,904 bytes.
mtp.fc.weight ([5120, 10240] BF16, 104,857,600 bytes) was then taken from the
corrected upstream revision and added as model-mtp-fc.safetensors, with the
index updated - 2399 tensors, 20,592,849,504 bytes. vLLM's own source
expects this tensor to be unquantized in NVFP4 checkpoints
(qwen3_5_mtp.py: "mtp.fc is stored as BF16 in NVFP4 checkpoints ... Force
unquantized"), so a spliced BF16 tensor is exactly what a fresh PTQ run would
have emitted.
The three VL processor configs (preprocessor_config.json,
processor_config.json, video_preprocessor_config.json) are included -
ModelOpt does not emit them and vLLM hard-fails a VL model at init without them.
min_p and logit_bias do not work under speculative decoding (vLLM warns at
startup).Qwen3_5MTP support that makes
the MTP head usable at all.