GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
QUASAR-QAT
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
QUASAR-QAT
Model Tree
google/gemma-4-12B-it
Input Modalities
Output Modalities
Supported Functionality
Higher point estimates on all four downstream benchmarks than Google's QAT model.
| Benchmark | QUASAR | Google QAT |
|---|---|---|
| MMLU | 70.4 | 67.8 |
| IFEval | 87.6 | 87.3 |
| ARC-Challenge | 51.3 | 50.1 |
| AGIEval | 24.2 | 22.6 |
| Average | 58.4 | 57.0 |
Same harness and backend for both checkpoints; the MMLU gain is +2.6 points (95% CI [+2.0, +3.2], paired bootstrap over 14,042 questions). Full configs, BF16 results and CIs are in EVAL.md.
Native QAT checkpoint: the INT4 codes come directly from training, rather than a post-training re-quantization. export_receipt.json verifies the export per tensor (328 quantized linears). The GGUF twin carries the same codes and scales and ships a --verify script that checks the two files against each other.
vllm serve QUASAR-QAT/gemma-4-12B-it-QUASAR-W4A16-G64 --dtype float16 --limit-mm-per-prompt '{"image":0}', then any OpenAI-compatible client. Text serving tested on vLLM 0.28.0 and 0.29.0 (--dtype float16 keeps the stored fp16 scales exact); 262,144-token context. For vision, use the GGUF twin with its projector.pack-quantized; everything else bf16. Full-attention layers share K/V (attention_k_eq_v), exported exactly as the model defines. 7.9 GB.QUASAR is loss-aware quantization-aware training. It improves the low-bit reconstruction during training, then exports to standard deployment formats with zero inference overhead.
This checkpoint was healed against the BF16 teacher for ~600M tokens: one epoch over ~377k prompts self-distilled from BF16 Gemma 4 12B-it with thinking on, fp32 master weights, INT4 group-64.
Based on Gemma 4 by Google DeepMind. Upstream LICENSE/NOTICE apply.
bibtex
@article{counathe2026quasar, title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction}, author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi}, journal={arXiv preprint arXiv:2608.13966}, year={2026}}@article{counathe2026quasar, title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction}, author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi}, journal={arXiv preprint arXiv:2608.13966}, year={2026}}