GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Shiftedx
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Shiftedx
Model Tree
Jackrong/Qwopus3.8-27B-Flash
Input Modalities
Output Modalities
Supported Functionality
This checkpoint does not support image or video input. The source model is multimodal, but this conversion intentionally removes its 333 vision tensors to reserve VRAM for the 27B language model, native MTP draft, and KV cache. SGLang supports Qwen vision models; this particular artifact does not.
44d24e8cb20ceb3cdf4fe200b5a0afd970ee748aThe export passed safetensors index/header validation, complete NVFP4 scale triplet checks, MTP equality checks, SGLang load validation, API generation, and long-context request tests.
The tested single-stream profile uses:
bash
python3 -m sglang.launch_server \--model-path /model \--served-model-name qwopus3.8-27b-nvfp4-mtp \--host 0.0.0.0 --port 1234 \--kv-cache-dtype fp8_e4m3 \--attention-backend flashinfer \--context-length 131072 \--max-total-tokens 129241 \--chunked-prefill-size 1024 \--mamba-radix-cache-strategy extra_buffer \--mamba-ssm-dtype bfloat16 \--max-mamba-cache-size 1 \--mem-fraction-static 0.96 \--max-running-requests 1 \--speculative-algorithm EAGLE \--speculative-draft-model-path /model \--speculative-num-steps 3 \--speculative-eagle-topk 1 \--speculative-num-draft-tokens 4 \--disable-radix-cache \--disable-prefill-cuda-graph \--weight-loader-drop-cache-after-load \--random-seed 42 \--reasoning-parser qwen3 \--tool-call-parser qwen3_coder
Do not add --language-only: the checkpoint already declares
language_model_only=true, and the extra runtime flag is incompatible with
this wrapper configuration in the qualified SGLang build.
Reproducible setup scripts and the pinned SGLang build are in
shiftedx/Qwen3.8-27B-RTX5090-SGLang-DFlash.
Measured on one RTX 5090 with one active request. Results depend on prompt and MTP acceptance and are not universal performance guarantees.
| Workload | Result |
|---|---|
| Recovered matched-prose harness, 3-run streaming decode median | 138.7 tok/s |
| 512-token synthetic runs after warm-up, end-to-end | 121.3–128.3 tok/s |
| 100,008-token prompt + 8 output tokens | 26.143 s |
| 128,008-token prompt + 8 output tokens | 40.365 s |
| Physical resident token pool | 129,241 tokens |
At the qualified 131K profile, GPU residency was approximately 27.6 GB and the SGLang container used approximately 4.9 GiB of host RAM. No CPU weight offload was enabled.
Apache-2.0, following the source checkpoint. Credit belongs to the Qwopus authors and the Qwen team; this repository only provides the quantized, text-only packaging and measured RTX 5090 serving profile.