Training
- Recipe:
configs/llm/qwen3_5_9b_nvfp4_qad.yaml at QATFactory commit 830bb757fd11aec0231b71f9ff600b17963bd886 (in the history of main), run on 4 GPUs with --train.gradient_accumulation_steps 4 (same global batch 16 as the recipe's 8 GPUs x 2), --model.attn_implementation sdpa, --train.save_steps 256.
- Data:
weili-0234/qatfactory-openperfectblend-100k-qwen3.5-9b-think at revision e1111a76c9321f1dc03b944dc39e36870db6f6c5 (file sha256 063674a0d6cd6a5a5714ca4b5eecfae9389b9a2b8d3de676d44f5b17c84c9cdf).
- Hardware and software: 1 node of 4x NVIDIA GB300, image vllm/vllm-openai:v0.30.0-aarch64 (sha256:4864d46625cbc3307623e29ac742030655e27249feba7b97ec925ce4cc4dfb56): torch 2.13.0+cu130, transformers 5.17.0, accelerate 1.15.0, plus flash-linear-attention 0.6.0 and datasets 5.0.0.
- Run: one epoch, 5,380 optimizer steps at global batch 16, lr 1e-6 cosine, forward KL to the frozen BF16 teacher, 8K context; two 4-hour jobs (resumed from checkpoint-3072), about 7 h of training at ~4.2 s/step. Final held-out (128 conversations) forward KL 0.0111 and teacher-student top-1 agreement 0.963.
Evaluation
Served with vLLM 0.30.0 (data-parallel 4 on GB300, native NVFP4 W4A4, max-model-len 65536) and evaluated with the released harness (scripts/eval/quality/, evalscope 1.9.1, temperature 1.0, top-p 0.95, top-k 20, 32K new tokens): GPQA-Diamond, Winogrande and HellaSwag (300), MMLU-Pro and MMMLU (21 per subset), AIME25 (5 samples per problem), LiveCodeBench v6 (300) and BigCodeBench (300 of 1,140). KL: forward KL to the BF16 model over top-20 prompt logprobs, 100 samples x 8K tokens each of UltraChat, the Pile and WikiText (scripts/eval/quality/run_kld.sh). The BF16 and RTN columns were measured with the same harness.
Table with columns: Benchmark, BF16, RTN NVFP4, This QAD, Paper QAD (W4A16-trained)| Benchmark | BF16 | RTN NVFP4 | This QAD | Paper QAD (W4A16-trained) |
|---|
| GPQA-Diamond | 78.8 | 73.7 | 76.3 | 77.8 |
| MMLU-Pro | 73.1 | 74.8 | 73.1 | 73.5 |
| MMMLU | 88.4 | 82.0 | 84.0 | 82.7 |
| Winogrande | 90.7 | 90.7 | 90.3 | 89.3 |
| HellaSwag | 75.0 | 70.0 | 71.7 | 72.0 |
| AIME25 | 50.0 | 38.0 | 45.3 | 43.3 |
| LiveCodeBench v6 | 87.3 | 62.7 | 80.0 | 83.0 |
| Average of the 7 above | 77.6 | 70.3 | 74.4 | 74.5 |
| BigCodeBench | 37.0 | 29.3 | 35.3 | 29.7 |
| KL UltraChat / Pile / WikiText | - | 0.0956 / 0.0573 / 0.0820 | 0.0810 / 0.0471 / 0.0691 | 0.0852 / 0.0484 / 0.0710 |
| KL mean | - | 0.0783 | 0.0657 | 0.0682 |
LiveCodeBench is graded with evalscope's local executor (no Docker on the evaluation cluster). BigCodeBench check programs are built by the harness unchanged and executed in an aarch64 rebuild of the official bigcodebench-evaluate image (all pinned requirements installed; no 4 GB memory cap; tensorflow does not import, which affects 1 of the 300 sampled tasks); its absolute scores run about 5 points above the paper's for every checkpoint, so compare within the column. Single-sample benchmarks of 198-300 questions carry 2-3 points of sampling noise.
Provenance
Files under provenance/: trainer_state.json (loss, held-out KL and top-1 history), kld-qad-nvfp4-r1.json (per-dataset KL), versions.txt (training environment).