GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ThakiCloud
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ThakiCloud
Model Tree
Qwen/Qwen3-30B-A3B
Input Modalities
Output Modalities
Supported Functionality
Single B200, vLLM 0.27.1, lm-eval 0.4.12, full sets, no --limit.
MMLU n=14,042 (loglikelihood); GSM8K-CoT n=1,319, 8-shot (generative).
Every row ran on the same GPU, engine and harness, so the columns are comparable to each other.
| build | recipe | size | MMLU | GSM8K (strict) |
|---|---|---|---|---|
Qwen/Qwen3-30B-A3B (bf16) | — | 56.89 GB | 0.7779 | 0.8741 |
| this model | llm-compressor, GPTQ | 18.11 GB | 0.7743 (−0.36pp) | 0.8878 |
ThakiCloud/Qwen3-30B-A3B-NVFP4-RTN | llm-compressor, RTN | 18.11 GB | 0.7676 (−1.03pp) | 0.8999 |
RedHatAI/Qwen3-30B-A3B-NVFP4 | RTN | 16.88 GB | 0.7675 (−1.04pp) | 0.8939 |
MMLU is statistically indistinguishable from bf16 (−0.36pp, z = −0.8). RTN loses about a full point (−1.03pp, z = −2.2). The GPTQ oneshot took 23,730 s against RTN's 5,271 s, so the extra 5.1 hours buys roughly 1pp of MMLU and nothing else we could measure. If that matters to you, take this build. If it does not, take the RTN one and save the compute.
The z values treat the two runs as independent, which is conservative: the eval set is shared, so the errors are correlated and the real significance is higher than stated.
We do not claim quantization improves math. GSM8K is higher than bf16 for every 4-bit build here, and the sign is consistent, but z runs 1.1 to 2.1 across 1,319 generative samples. That is not separable from sampling noise. The defensible statement is no measurable degradation on GSM8K, and nothing stronger.
We also reproduce RedHatAI/Qwen3-30B-A3B-NVFP4 to within 0.01pp MMLU with our own RTN build,
which is how we established that their NVFP4 is RTN rather than GPTQ.
Both timings come from quantize_meta.json, which ships in this repo, so you can check
them against the card rather than take our word for it.
A 4-bit checkpoint can hold NVFP4 weights and still expand them to bf16 for every matmul. That path is slower than bf16 and saves only memory. So we assert the kernel rather than assume it:
markdown
Using 'FLASHINFER_TRTLLM' NvFp4 MoE backend # B200, no fallback warning
Verified on B200 (SM100) with vLLM 0.27.1. No does not have native support for FP4 warning
appeared in the serving log.
⚠️ This is Blackwell-only. On Hopper (H200/H100) vLLM falls back to weight-only Marlin emulation, which we measured separately at roughly 0.85x of bf16 — that is, slower. NVFP4 is not a speed win on SM90 and we would not recommend it there. On SM90 the format that wins is FP8.
Single B200, decode-dominant workload, GPU power only (host and cooling excluded):
| build | tok/J |
|---|---|
| bf16 | 27.4 |
| RTN NVFP4 | 46.8 |
⚠️ Throughput in that run was not saturated (GPU utilisation 54 to 85 percent, 29-token prompts), so we withdraw any throughput claim from it and quote only the power gap. A later saturated run on a different model in this family measured NVFP4 at 1.07x to 1.32x of FP8 across the concurrency band; that number belongs to that model, not this one.
bash
vllm serve ThakiCloud/Qwen3-30B-A3B-NVFP4-GPTQ --max-model-len 8192
Check your serving log for the backend line above. If you see MARLIN or a native-support
warning, you are measuring emulation, not FP4.
One model, one recipe, one engine version, one GPU generation for the kernel claim. Quality was measured on MMLU and GSM8K only; we have not evaluated code, multilingual or long-context behaviour, and we do not claim anything about them.