GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
gittensor-model-hub
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
gittensor-model-hub
Model Tree
gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090
Input Modalities
Output Modalities
Supported Functionality
It saves download size, not memory. Measured on an RTX 5090, SGLang reports mem usage=18.80 GB for weights with and without the head — byte-for-byte the same, because mtp.* is namespaced outside the target graph and is never allocated unless you turn speculation on. If you are hoping to free VRAM, this variant will not do that.
It removes speculative decoding on vLLM entirely. DSpark is a SGLang-only algorithm, so on vLLM the native MTP head is the only draft path. Without it you decode at ~81.6 tok/s instead of ~136.9.
So this variant makes sense if you are SGLang-only and using the DSpark drafter (which is both faster and smaller than MTP anyway), or if you are shipping the weights somewhere that 0.85 GB matters. Otherwise use the parent — the head is free when idle.
Even on the parent checkpoint, the MTP head is not the best option:
| Drafter | Decode | Accept | Size |
|---|---|---|---|
| DSpark-NVFP4 v2 (SGLang) | 155.8 tok/s | 2.886 | 1.41 GB |
| Native MTP head (SGLang or vLLM) | 136.9 tok/s | 2.758 | 5.53 GB |
| No speculation | 81.6 tok/s | — | — |
DSpark wins on speed, acceptance and size — the MTP head's value is that it works on vLLM, where DSpark does not exist.
SGLang with the DSpark drafter (recommended for this variant):
bash
sglang serve --model-path gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090-No-MTP \--trust-remote-code --tp-size 1 \--context-length 65536 --kv-cache-dtype fp8_e4m3 \--attention-backend flashinfer --chunked-prefill-size 2048 \--mamba-radix-cache-strategy extra_buffer_lazy --mamba-ssm-dtype bfloat16 \--mem-fraction-static 0.90 --max-running-requests 2 \--speculative-algorithm DSPARK \--speculative-draft-model-path gittensor-model-hub/Qwen3.8-27B-DSpark-NVFP4 \--speculative-dspark-block-size 7 --speculative-draft-model-quantization modelopt_fp4 \--reasoning-parser qwen3 --tool-call-parser qwen3_coder
vLLM for the full 262,144-token window (no speculation available on this variant):
bash
vllm serve gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090-No-MTP \--quantization modelopt --kv-cache-dtype fp8 --trust-remote-code \--max-model-len 262144 --max-num-seqs 16 --gpu-memory-utilization 0.97 \--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml
Do not pass --speculative-config '{"method":"mtp"}' to this variant — the head is not present.
Removed 15 tensors, all under the mtp. prefix:
markdown
mtp.fc.weight mtp.layers.0.self_attn.{q,k,v,o}_proj.weightmtp.norm.weight mtp.layers.0.self_attn.{q,k}_norm.weightmtp.pre_fc_norm_embedding.weight mtp.layers.0.mlp.{gate,up,down}_proj.weightmtp.pre_fc_norm_hidden.weight mtp.layers.0.{input,post_attention}_layernorm.weight
They were BF16, never quantized (the parent's hf_quant_config.json already excluded mtp*). Alongside the tensor removal:
config.json — text_config.mtp_num_hidden_layers 1 → 0; mtp_use_dedicated_embeddings droppedhf_quant_config.json — exclude_modules 149 → 147 (mtp*, mtp.layers.0* removed)mtp.* and became empty), index and crc32.txt regeneratedEverything else — tokenizer, chat template, preprocessor configs, generation config — is copied unchanged from the parent.
Verified: loads in SGLang as Qwen3_5ForConditionalGeneration, quant_algo=NVFP4, and answers correctly.
Unchanged from the parent, since the target weights are identical. See the parent model card for the full quantization recipe, 262K context behaviour, accuracy smoke and chat-template documentation.
Apache 2.0, same as the parent and the Qwen3.8-27B base model.