Evaluation
Thinking mode, pass@1 averaged over 1–32 seeds, same settings for all models.
Usage
Runs on vLLM on NVIDIA Hopper GPUs only (H100 / H200, sm90): the W4A8 kernel in vLLM requires compute capability 9.0.
vLLM:
vllm serve qda-lab/QDA-Qwen3.6-35B-A3B-W4A8
How it was made
Weights are 4-bit integers in groups of 128 and activations are 8-bit floating point (dynamic), quantized with GPTQ on a Hadamard-rotated copy of the model; the rotation is folded
into the weights, so there is no runtime transform.
MoE routers, shared experts and the output head stay in BF16. The recipe was chosen by QDA's automated search.
Reproduce
hf download qda-lab/qda-recipe-inputs --repo-type dataset --include "QDA-Qwen3.6-35B-A3B-W4A8/*" --local-dir inputsqda build inputs/QDA-Qwen3.6-35B-A3B-W4A8/qwen3.6-35b-a3b-w4a8.yaml --set residency=stream
Needs the QDA library (the qda command). Recipe: qwen3.6-35b-a3b-w4a8.yaml; inputs
(calibration sample and distillation data): QDA-Qwen3.6-35B-A3B-W4A8/ in qda-lab/qda-recipe-inputs.
Takes about 50 min on 1× H100 (measured). The rebuild matches this checkpoint's KL to BF16 within 5 %; weights are not bit-identical.
License
Apache-2.0, the same as the base model.
Citation
TBD.