Evaluation
Thinking mode, pass@1 averaged over 1–32 seeds, same settings for all models.
Usage
Runs on vLLM only (SGLang does not support this 3-bit format yet), on NVIDIA H100, A100, L40S, RTX Pro 6000 and B200 GPUs.
vLLM:
vllm serve qda-lab/QDA-Qwen3.5-9B-W3A16
How it was made
Weights are 3-bit integers in groups of 128, with the output head and token embeddings at 4 bits; the group scales
and norm tensors were then fine-tuned by distillation. The recipe was chosen by QDA's automated search.
Reproduce
hf download qda-lab/qda-recipe-inputs --repo-type dataset --include "QDA-Qwen3.5-9B-W3A16/*" --local-dir inputsqda build inputs/QDA-Qwen3.5-9B-W3A16/qwen3.5-9b-w3a16.yaml
Needs the QDA library (the qda command). Recipe: qwen3.5-9b-w3a16.yaml; inputs
(the 3-bit start checkpoint in start/, 6.7 GB, and the distillation data): QDA-Qwen3.5-9B-W3A16/ in qda-lab/qda-recipe-inputs.
Takes about 4.5 h on 1× H100 (measured). The rebuild matches this checkpoint's KL to BF16 within 5 %; weights are not bit-identical.
The recipe starts from a 3-bit checkpoint we quantized from Qwen3.5-9B with the official GSQ code (IST-DASLab/GSQ, paper); the dataset folder holds it (start/).
License
Apache-2.0, the same as the base model.
Citation
TBD.