GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ucbye
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ucbye
Model Tree
saricles/Qwen3-Coder-Next-NVFP4-GB10
Input Modalities
Output Modalities
Supported Functionality
Qwen/Qwen3-Coder-Next by Alibaba / Qwen team (Apache-2.0)saricles using LLM Compressor with LLMCOMPRESSOR_MOE_CALIBRATE_ALL_EXPERTS=1 (all 512 experts calibrated)HuggingFaceH4/ultrachat_200k (64 samples × 2048 tok)qwen3_next — Hybrid DeltaNet linear attention + full attention + latent MoEcompressed-tensors; lm_head, embed_tokens, linear_attn layers, mlp.gate, mlp.shared_expert_gate kept in BF16Measured on DGX Spark (GB10 Blackwell, SM12.1, 128 GB unified LPDDR5X) with vLLM (Marlin NVFP4, FP8 KV cache):
| Metric | Value |
|---|---|
| Throughput (single request) | ~61 tok/s |
| KV cache concurrency @ 262K | ~5 concurrent sessions (DeltaNet has no KV cache → 31.65× vs a pure-attention 80B model) |
Same decode throughput as Nemotron-3-Nano-30B-A3B-NVFP4 (both 3B active), but with a native 262K context window.
bash
docker run --rm --runtime=nvidia --gpus all \ -p 8000:8000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ -e VLLM_NVFP4_GEMM_BACKEND=marlin \ -e VLLM_TEST_FORCE_FP8_MARLIN=1 \ -e VLLM_USE_FLASHINFER_MOE_FP4=0 \ -e VLLM_MARLIN_USE_ATOMIC_ADD=1 \ vllm/vllm-openai:cu130-nightly \ gdubicki/Qwen3-Coder-Next-NVFP4-GB10 \ --dtype auto \ --kv-cache-dtype fp8 \ --gpu-memory-utilization 0.90 \ --max-model-len 262144 \ --attention-backend flashinfer \ --tool-call-parser qwen3_coder \ --enable-chunked-prefill \ --enable-prefix-cachingFull deployment scripts: https://github.com/grzegorzdubicki-ai/qwen3-coder-next
docker run --rm --runtime=nvidia --gpus all \ -p 8000:8000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ -e VLLM_NVFP4_GEMM_BACKEND=marlin \ -e VLLM_TEST_FORCE_FP8_MARLIN=1 \ -e VLLM_USE_FLASHINFER_MOE_FP4=0 \ -e VLLM_MARLIN_USE_ATOMIC_ADD=1 \ vllm/vllm-openai:cu130-nightly \ gdubicki/Qwen3-Coder-Next-NVFP4-GB10 \ --dtype auto \ --kv-cache-dtype fp8 \ --gpu-memory-utilization 0.90 \ --max-model-len 262144 \ --attention-backend flashinfer \ --tool-call-parser qwen3_coder \ --enable-chunked-prefill \ --enable-prefix-caching