Training
- Recipe:
configs/llm/qwen3_5_9b_nvfp4_qad_w4a4.yaml at QATFactory commit 4f911712810d65c6f186195647a5a89c3108f351 (in the history of main), run on 4 GPUs with --train.gradient_accumulation_steps 4 (same global batch 16 as the recipe's 8 GPUs x 2), --model.attn_implementation sdpa, --train.save_steps 256.
- Data:
qatfactory/openperfectblend-100k-qwen3.5-9b-think at revision 19edf87f0eed00f972bb277764a8eca5d0fe8031 (file sha256 063674a0d6cd6a5a5714ca4b5eecfae9389b9a2b8d3de676d44f5b17c84c9cdf).
- Hardware and software: 1 node of 4x NVIDIA GB300, image vllm/vllm-openai:v0.30.0-aarch64 (sha256:4864d46625cbc3307623e29ac742030655e27249feba7b97ec925ce4cc4dfb56): torch 2.13.0+cu130, transformers 5.17.0, accelerate 1.15.0, plus flash-linear-attention 0.6.0 and datasets 5.0.0.
- Run: one epoch, 5,380 optimizer steps at global batch 16, lr 1e-6 cosine, forward KL to the frozen BF16 teacher, 8K context, weights and activations fake-quantized to NVFP4; two 4-hour jobs resumed from checkpoint. Final held-out (128 conversations) forward KL 0.0325 and teacher-student top-1 agreement 0.936.
Evaluation
Served with vLLM 0.30.0 (data-parallel 4 on GB300, native NVFP4 W4A4, max-model-len 65536) and evaluated with the released harness (scripts/eval/quality/, evalscope 1.9.1, temperature 1.0, top-p 0.95, top-k 20, 32K new tokens): GPQA-Diamond, Winogrande and HellaSwag (300), MMLU-Pro and MMMLU (21 per subset), AIME25 (5 samples per problem), LiveCodeBench v6 (300) and BigCodeBench (300 of 1,140). KL: forward KL to the BF16 model over top-20 prompt logprobs, 100 samples x 8K tokens each of UltraChat, the Pile and WikiText (scripts/eval/quality/run_kld.sh). The BF16 column was measured with the same harness.
Table with columns: Benchmark, BF16, This QAD, Paper QAD (W4A4-trained)| Benchmark | BF16 | This QAD | Paper QAD (W4A4-trained) |
|---|
| GPQA-Diamond | 78.8 | 76.3 | 78.3 |
| MMLU-Pro | 73.1 | 72.5 | 71.4 |
| MMMLU | 88.4 | 84.4 | 83.7 |
| Winogrande | 90.7 | 89.0 | 89.0 |
| HellaSwag | 75.0 | 74.7 | 75.0 |
| AIME25 | 50.0 | 51.3 | 47.3 |
| LiveCodeBench v6 | 87.3 | 80.0 | 78.3 |
| Average of the 7 above | 77.6 | 75.5 | 74.7 |
| BigCodeBench | 37.0 | 34.3 | 28.0 |
| KL UltraChat / Pile / WikiText | - | 0.1160 / 0.0517 / 0.0750 | 0.1008 / 0.0507 / 0.0723 |
| KL mean | - | 0.0809 | 0.0746 |
LiveCodeBench is graded with evalscope's local executor (no Docker on the evaluation cluster). BigCodeBench check programs are built by the harness unchanged and executed in an aarch64 rebuild of the official bigcodebench-evaluate image (all pinned requirements installed; no 4 GB memory cap; tensorflow does not import, which affects 1 of the 300 sampled tasks); its absolute scores run about 5 points above the paper's for every checkpoint, so compare within the column. Single-sample benchmarks of 198-300 questions carry 2-3 points of sampling noise. The BF16 column is the same BF16 run for every checkpoint in this series.
Provenance
Files under provenance/: trainer_state.json (loss, held-out KL and top-1 history), kld-q35-nvfp4-w4a4-qad.json (per-dataset KL), versions.txt (training environment).