GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
QUASAR-QAT
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
QUASAR-QAT
Model Tree
Qwen/Qwen3.5-4B
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "vllm>=0.26"vllm serve QUASAR-QAT/Qwen3.5-4B-QUASAR-NVFP4-W4A4 --max-model-len 262144 --reasoning-parser qwen3
Target runtime is Blackwell (compute capability 10.0+), where vLLM uses the native NVFP4 W4A4 path. On Hopper, vLLM falls back to weight-only Marlin; we verified identical greedy outputs to the W4A16 release on H100.
Per-token forward KL(BF16 ‖ quantized) and top-1 agreement on 734 held-out prompts of the BF16 model's own responses (934k tokens), with vLLM's NVFP4 activation quantization applied to the inputs of the same 200 projections; each W4A4 row uses its own calibrated scales. Methodology and audit trail: EVAL.md.
| KL ↓ | KL p99 ↓ | Top-1 ↑ | |
|---|---|---|---|
| QUASAR W4A4 (this model) | .046 | .357 | 93.3% |
| cosmicproc W4A4 | .075 | .587 | 91.4% |
| BF16 weights + FP4 activations | .037 | .265 | 94.1% |
FP4 activations account for most of the remaining gap to BF16: the activation-only reference is .037 KL, while QUASAR W4A4 is .046. Under the same evaluation harness, cosmicproc is .075.
Weights: NVFP4 on all 200 major decoder projections; bit-exact to the QUASAR-trained W4A16 weights. Activations: NVFP4 group-16 with dynamic local scales and calibrated static per-tensor global scales. BF16: embeddings, norms, conv1d kernels, small delta-net gate vectors, vision tower and MTP drafter.
QUASAR-QAT/Qwen3.5-4B-QUASAR-Q4_0-GGUF — native Q4_0 QAT for llama.cpp / Ollama / LM Studio.QUASAR-QAT/Qwen3.5-4B-QUASAR-NVFP4 — the W4A16 release with the full benchmark head-to-head.QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 — the 27B W4A4 sibling.bibtex
@article{counathe2026quasar,title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},journal={arXiv preprint arXiv:2608.13966},year={2026}}