GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
QUASAR-QAT
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
QUASAR-QAT
Model Tree
google/gemma-4-12B-it
Input Modalities
Output Modalities
Supported Functionality
This checkpoint also beats Google's W4A16 QAT across the benchmarks below (same harness, same backend, every row).
| Benchmark | QUASAR | Google QAT |
|---|---|---|
| MMLU | 70.4 | 67.8 |
| IFEval | 87.6 | 87.3 |
| ARC-Challenge | 51.3 | 50.1 |
| AGIEval | 24.2 | 22.6 |
| Average | 58.4 | 57.0 |
The MMLU gain is statistically robust: +2.6 points, 95% CI [+2.0, +3.2] (paired bootstrap over 14,042 questions).
These are the trained INT4 codes and per-group fp16 scales, recovered bit-exactly from the training checkpoint — 328 quantized linears, per-tensor receipt in export_receipt.json. No post-hoc rounding.
The GGUF twin ships a --verify script that checks the two files against each other byte for byte.
vllm serve QUASAR-QAT/gemma-4-12B-it-QUASAR-W4A16-G64 --dtype float16, then any OpenAI-compatible client (text).AutoModelForCausalLM.from_pretrained("QUASAR-QAT/gemma-4-12B-it-QUASAR-W4A16-G64", dtype=torch.bfloat16) with compressed-tensors installed (this is how the fidelity numbers above were measured)."chat_template_kwargs": {"enable_thinking": false} to turn it off.328 decoder linears int4 symmetric, group_size 64, fp16 scales, compressed-tensors pack-quantized; everything else bf16. Full-attention layers share K/V (attention_k_eq_v), exported exactly as the model defines. Loads wherever Google's W4A16 release loads.
| File | Size |
|---|---|
model-0000{1,2}-of-00002.safetensors — W4A16 g64 weights plus bf16 embeddings, norms and vision/audio towers | 7,910,339,296 bytes (7.9 GB) |
export_receipt.json — per-tensor bit-exactness receipt · EVAL.md · scripts/ (packer/verifier, eval harvest, paired bootstrap) |
QUASAR is loss-aware quantization-aware training. It improves the low-bit reconstruction during training, then exports to standard deployment formats with zero inference overhead.
This checkpoint was healed against the BF16 teacher for ~600M tokens: one epoch over ~377k prompts self-distilled from BF16 Gemma 4 12B-it with thinking on, fp32 master weights, INT4 group-64.
Paper: arXiv:2608.13966
Based on Gemma 4 by Google DeepMind. Upstream LICENSE/NOTICE apply.
bibtex
@article{counathe2026quasar,title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},journal={arXiv preprint arXiv:2608.13966},year={2026}}