GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
QUASAR-QAT
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
QUASAR-QAT
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
Compatible with vLLM, with no conversion step:
bash
pip install "vllm>=0.27"vllm serve QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 \--max-model-len 262144 \--gpu-memory-utilization 0.85 \--speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'
On a 32 GB card such as an RTX 5090, lower the context to --max-model-len 65536.
Requires an NVIDIA GPU with FP4 support (Blackwell, compute capability 10.0+).
Two other public NVFP4 builds of this model, evaluated under the same setup (GPQA-Diamond: 2 runs, n=396; AIME'26: 3 repeats, n=90). Bold = best among the NVFP4 builds.
| Model | Size | NVFP4 linears | GPQA-D | AIME'26 |
|---|---|---|---|---|
| QUASAR (this model) | 19.7 GB | 496/496 | 90.91 | 100.0 |
| BF16 original | 55.6 GB | — | 91.41 | 100.0 |
| Unsloth NVFP4 | 23.4 GB | 168/496 | 89.39 | 97.78 |
| Inferact NVFP4 | 26.4 GB | 304/496 | 87.63 | 96.67 |
One epoch of loss-aware NVFP4 quantization-aware distillation against the frozen BF16 teacher: global batch size 32, learning rate 1e-6, 2446 steps.
bibtex
@article{counathe2026quasar,title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},journal={arXiv preprint arXiv:2608.13966},year={2026}}