Qwen3.8-27B-QUASAR-NVFP4 API & Inference Endpoint

How to run

Compatible with vLLM, with no conversion step:

bash

pip install "vllm>=0.27" vllm serve QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 \  --max-model-len 262144 \  --gpu-memory-utilization 0.85 \  --max-num-seqs 512 \  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}' \  --enable-auto-tool-choice --tool-call-parser qwen3_xml \  --reasoning-parser qwen3

On a 32 GB card such as an RTX 5090, lower the context to --max-model-len 65536.

Requires an NVIDIA GPU with FP4 support (Blackwell, compute capability 10.0+).

Quality and size comparison

Two other public NVFP4 builds of this model, evaluated under the same setup (GPQA-Diamond: 2 runs, n=396; AIME'26: 3 repeats, n=90). Bold = best among the NVFP4 builds.

Table with columns: Model, Size, NVFP4 linears, GPQA-D, AIME'26
Model	Size	NVFP4 linears	GPQA-D	AIME'26
QUASAR (this model)	19.7 GB	496/496	90.91	100.0
BF16 original	55.6 GB	—	91.41	100.0
Unsloth NVFP4	23.4 GB	168/496	89.39	97.78
Inferact NVFP4	26.4 GB	304/496	87.63	96.67

Table with columns: Model, MRCR, MRCR 8-needle, HMMT'26, SuperGPQA, LiveCodeBench, RULER 32K, RULER 64K, RULER 128K
Model	MRCR	MRCR 8-needle	HMMT'26	SuperGPQA	LiveCodeBench	RULER 32K	RULER 64K	RULER 128K
QUASAR (this model)	79.1	52.7	73.4	62.4	79.2	96.3	96.2	95.8

Training

One epoch of loss-aware NVFP4 quantization-aware distillation against the frozen BF16 teacher: global batch size 32, learning rate 1e-6, 2446 steps.

Citation

arxiv.org/abs/2608.13966

bibtex

@article{counathe2026quasar,  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},  journal={arXiv preprint arXiv:2608.13966},  year={2026}}

How to run

Compatible with vLLM, with no conversion step:

bash

pip install "vllm>=0.27" vllm serve QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 \  --max-model-len 262144 \  --gpu-memory-utilization 0.85 \  --max-num-seqs 512 \  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}' \  --enable-auto-tool-choice --tool-call-parser qwen3_xml \  --reasoning-parser qwen3

On a 32 GB card such as an RTX 5090, lower the context to --max-model-len 65536.

Requires an NVIDIA GPU with FP4 support (Blackwell, compute capability 10.0+).

Quality and size comparison

Two other public NVFP4 builds of this model, evaluated under the same setup (GPQA-Diamond: 2 runs, n=396; AIME'26: 3 repeats, n=90). Bold = best among the NVFP4 builds.

Table with columns: Model, Size, NVFP4 linears, GPQA-D, AIME'26
Model	Size	NVFP4 linears	GPQA-D	AIME'26
QUASAR (this model)	19.7 GB	496/496	90.91	100.0
BF16 original	55.6 GB	—	91.41	100.0
Unsloth NVFP4	23.4 GB	168/496	89.39	97.78
Inferact NVFP4	26.4 GB	304/496	87.63	96.67

Table with columns: Model, MRCR, MRCR 8-needle, HMMT'26, SuperGPQA, LiveCodeBench, RULER 32K, RULER 64K, RULER 128K
Model	MRCR	MRCR 8-needle	HMMT'26	SuperGPQA	LiveCodeBench	RULER 32K	RULER 64K	RULER 128K
QUASAR (this model)	79.1	52.7	73.4	62.4	79.2	96.3	96.2	95.8

Training

One epoch of loss-aware NVFP4 quantization-aware distillation against the frozen BF16 teacher: global batch size 32, learning rate 1e-6, 2446 steps.

Citation

arxiv.org/abs/2608.13966

bibtex

@article{counathe2026quasar,  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},  journal={arXiv preprint arXiv:2608.13966},  year={2026}}

Qwen3.8-27B-QUASAR-NVFP4

README

How to run

Quality and size comparison

Training

Citation

Explore FriendliAI today

README

How to run

Quality and size comparison

Training

Citation