Evaluation
Table with columns: Benchmark, Problems × draws, Output cap, BF16 (19.3 GB), QDA W4A4 (9.4 GB), RedHatAI NVFP4 (9.4 GB)| Benchmark | Problems × draws | Output cap | BF16 (19.3 GB) | QDA W4A4 (9.4 GB) | RedHatAI NVFP4 (9.4 GB) |
|---|
| AIME 2025 | 30 × 32 | 81,920 | 85.3 | 80.5 | 76.9 |
| HMMT Feb 2026 | 33 × 32 | 81,920 | 68.9 | 64.0 | 59.8 |
| GPQA-Diamond | 198 × 16 | 81,920 | 82.2 | 78.9 | 77.4 |
| IFBench | 300 × 16 | 81,920 | 66.9 | 65.7 | 62.5 |
| MMLU-Pro | 12,032 × 1 | 32,768 | 81.0 | 79.8 | 79.5 |
| LiveCodeBench | 454 × 4 | 81,920 | 71.1 | 64.7 | 61.2 |
| Average | | | 75.9 | 72.3 | 69.5 |
Thinking mode, pass@1 averaged over the draws; sampling temperature 1.0, top-p 0.95, top-k 20, presence-penalty 1.5. Same problems, seeds, sampling and output caps for every model.
Usage
FP4 runs natively on NVIDIA Blackwell GPUs (B200, GB300, RTX PRO 6000). Evaluated with vLLM 0.33.0 on GB300.
vllm serve qda-lab/QDA-Qwen3.5-9B-W4A4
How it was made
Every decoder linear is NVFP4: E2M1 weights with one E4M3 scale per 16 inputs and an FP32 tensor scale,
activations quantized the same way at run time. The Gated DeltaNet in_proj_a / in_proj_b, lm_head, the
embeddings and the vision tower stay in BF16. Weights were quantized with GPTQ (static act-order, an
FP4-aware MSE clip). The search kept the size of the plain NVFP4 build and chose the method.
Reproduce
Install QDA 0.1.0 (Linux, an NVIDIA GPU and uv):
git clone https://github.com/humanfia/qda.git && cd qdauv venv -p 3.12 --managed-python && . .venv/bin/activateuv pip install -c constraints.txt "./packages/qda-quant[hf,gguf]" "./packages/qda-kl[vllm]" ./packages/qda-agent
Then:
hf download qda-lab/qda-recipe-inputs --repo-type dataset --include "QDA-Qwen3.5-9B-W4A4/*" --local-dir inputscd inputs/QDA-Qwen3.5-9B-W4A4qda build qwen3.5-9b-w4a4-nvfp4.yaml --set out=QDA-Qwen3.5-9B-W4A4-rebuild --set "devices=[0]"
Needs QDA 0.1.0 and one Blackwell GPU. Original model: Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a, the revision the recipe builds from. On a GB300, a rebuild from this recipe was byte-identical to this checkpoint. A rebuild on one GB300 takes about 15 min.
License
Apache-2.0, the same as the base model.
Citation
QDA: Closed-Loop LLM Quantization Design with Agents
@article{li2026qda, title = {QDA: Closed-Loop LLM Quantization Design with Agents}, author = {Li, Xingyang and Zhao, Tianlang and Han, Song and Zhu, Ligeng}, journal = {arXiv preprint arXiv:2611.23388}, year = {2026}}