How it was made
Stage 1, REAP expert pruning. llm-compressor 0.13.0 REAPPruningModifier on bf16 Kimi-K2.5 (decompressed from the INT4 release), mixed calibration corpus, 512 × 4096 tokens, layers pruned sequentially; 152 of 384 routed experts removed per layer. The pruned checkpoint is the base bf16 weights with the dropped experts removed and the router / score-correction bias sliced (REAP only selects experts); the per-layer kept indices are in reap_kept_experts.jsonl.
Stage 2, dense NVFP4 W4A4. MoE-SQ GPTQ initialisation with NVFP4 values, group size 16, activation fake-quant, refinement off, mixed calibration, 512-sample Hessians, percdamp 0.1 — the same config as kwanhee/Kimi-K2.5-REAP50-NVFP4-W4A4-GS16 with only the model path changed (moe_sq_config.yaml).
Notes:
- Stage 1 and the slicing ran on 8× NVIDIA B300 (SM103).
- Stage 2 ran with eager attention (an explicit causal mask) instead of FlashAttention-2, which is not available for SM103 in this environment; this changes only floating-point summation order in the calibration forward passes.
Serving
Same stack as the other MoE-SQ / REAP Kimi checkpoints: vLLM v0.30.0 with the MoE-SQ patch
(IST-DASLab/MoESQ, integrations/vllm/install.sh). Multimodal paths are untested; all evaluations are text-only.
License and attribution
Base model moonshotai/Kimi-K2.5 under a Modified MIT License
(see LICENSE, which applies to this checkpoint); upstream notices in THIRD_PARTY_NOTICES.md.
REAP is due to Cerebras Research; the implementation used is llm-compressor's REAPPruningModifier.