GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
systemsofrecord
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
systemsofrecord
Model Tree
Qwen/Qwen3-Coder-30B-A3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Quantized → INT4 (g128, symmetric) | Kept in BF16 |
|---|---|
| all 128 routed experts × 48 layers | token embeddings, lm_head |
attention q/k/v/o projections | MoE router gates, all norms |
Only transformer Linear weights are quantized; the embedding, output head, router gates,
and norms stay BF16 for quality. It remains a standard Qwen3MoeForCausalLM — full GQA
attention, SwiGLU, 128 experts / 8 active — so it uses vLLM's mainstream MoE path.
bash
vllm serve systemsofrecord/Qwen3-Coder-30B-A3B-Instruct-W4A16 \--tensor-parallel-size 2 \--dtype float16 \--max-model-len 32768 \--gpu-memory-utilization 0.90 \--enable-auto-tool-choice \--tool-call-parser qwen3_coder
Qwen3MoeForCausalLM support (≥ 0.25).Verified: loaded and generated correct code on 2× NVIDIA A2 under vLLM 0.25.1 — ~7.9 GB weights/GPU at TP=2, Marlin wNa16 MoE kernel, CUDA graphs captured cleanly.
W4A16 — weights 4-bit int, group_size=128, symmetric; activations unquantized.pack-quantized (INT4 packed into INT32 + group scales).lm_head, embed_tokens, MoE router gates, norms.Apache-2.0, inherited from the base model Qwen/Qwen3-Coder-30B-A3B-Instruct. This repository only redistributes a quantized copy of those weights; all model capabilities and credit belong to the Qwen team.