What this is
Same REAP-50%-pruned source checkpoint as the primary release, quantized
to the identical NVFP4A16 scheme (weight-only, 4-bit) — but with GPTQ
(Hessian-based, error-correcting) rounding instead of the primary release's
plain round-to-nearest (RTN). Same size, same architecture, same serving
kernel. The only variable changed is the quantization algorithm.
12.4512 GiB — exact byte-for-byte size match to the primary release.
Why this was tried
GPTQ is documented in current literature to generally outperform RTN for
NVFP4 weight recovery. It was tested here specifically to see whether that
general finding held for this checkpoint and closed any of the gap to the
competitive bar (Devstral Small 2512, 56.4% SWE-bench Verified under the
same mini-swe-agent bash-only scaffold).
Result: no measurable difference from the primary release
Paired McNemar test (not just comparing point estimates) against the
primary release's own accuracy suite, same evaluation methodology:
Table with columns: benchmark, primary (RTN), this build (GPTQ), discordant pairs, McNemar p| benchmark | primary (RTN) | this build (GPTQ) | discordant pairs | McNemar p |
|---|
| HumanEval+ | 90.85% | 89.63% | 6 / 164 | 0.68 (not significant) |
| MBPP+ | 89.95% | 89.42% | 18 / 378 | 0.81 (not significant) |
Both comparisons are underpowered (well under the ~40-50 discordant pairs
needed for adequate statistical power) — this is genuinely "no detectable
difference," not confident evidence of true equivalence, and not evidence
GPTQ is worse either. The run itself was clean: 5h23min, zero exceptions,
zero fallback-to-RTN warnings across all 15,520 quantized modules.
Practical read: this specific checkpoint's RTN baseline appears to
already be close to its own accuracy ceiling, leaving little room for a
better rounding algorithm to show up. GPTQ's general advantage over RTN
(well-supported in the literature for other models) doesn't disappear —
it just isn't visible here at the precision this evaluation can resolve.
Full writeup, including the literature verification done before running
this and the exact promotion criteria this didn't clear:
ROADMAP.md
in the primary release's GitHub repo.
Quantization details
Table with columns: Field, Value| Field | Value |
|---|
| Base | Same REAP-50%-pruned checkpoint as the primary release |
| Quantization method | llm-compressor GPTQModifier — actorder="static", block_size=128, dampening_frac=0.01 |
| Quantization scheme | NVFP4A16 — weight-only, identical to the primary release |
| Calibration | evol-codealpaca (same set, same seed as the primary release, isolating the algorithm as the only variable) |
License
Apache 2.0, inherited from the base model Kwaipilot/KAT-Coder-V2.5-Dev.