tomhu

Qwen3-Coder-Next-SWE-Prune-26B-A3B

README

License: apache-2.0

What changed

Routed experts per MoE layer: 512 → 154 (30.08% retained; 69.92% pruned).
Activated routed experts per token: 10 (unchanged).
Router rows were remapped to the retained experts.
Dense, attention, shared-expert, tokenizer, and chat-template weights are unchanged.
No fine-tuning was performed after pruning.
Experts were selected independently in each layer using importance scores computed from successful, submitted SWE-style agent trajectories.

The checkpoint uses the official Qwen FP8 block-quantized format and is directly loadable by recent vLLM/Transformers versions that support Qwen3NextForCausalLM.

Parameter count

Quantization scale tensors are excluded from learned-parameter counts.

Table with columns: Metric, Exact count, Model-name convention
Metric	Exact count	Model-name convention
Total learned parameters	25,593,035,520	26B
Activated parameters per token	3,849,763,584	A3B
Routed-expert parameters	23,253,221,376	—
Always-active parameters	2,339,814,144	—

The A3B label follows the upstream Qwen convention. Pruning reduces stored parameters while keeping 10 routed experts active per token.

Architecture

Layers: 48
Hidden size: 2,048
Retained routed experts per layer: 154
Activated routed experts per token: 10
Shared experts per layer: 1
Expert intermediate size: 512
Native context length: 262,144
Inference mode: non-thinking

SWE-bench Verified evaluation

The complete 500-instance SWE-bench Verified evaluation used the unmodified official SWE-bench harness and the official mini-swe-agent prompt, with 250 steps, temperature=1.0, top_p=0.95, and top_k=40:

308/500 resolved (61.6%)
130 unresolved
38 empty patches
24 harness errors

vLLM example

bash
vllm serve tomhu/Qwen3-Coder-Next-SWE-Prune-26B-A3B \
  --tensor-parallel-size 1 \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --max-num-seqs 24 \
  --max-num-batched-tokens 8192 \
  --enable-prefix-caching \
  --generation-config vllm

Recommended sampling parameters:

json
{
  "temperature": 1.0,
  "top_p": 0.95,
  "top_k": 40,
  "chat_template_kwargs": {"enable_thinking": false}
}

Provenance and audit files

The repository includes the retained-expert mapping, pruning manifest, source revision metadata, tensor-level validation results, and shard checksums.

Upstream revision: Qwen/Qwen3-Coder-Next-FP8@da6e2ed27304dd39abadd9c82ef50e8de67bdd4c

License

This derivative checkpoint follows the upstream Apache-2.0 license. See the upstream model card for the original model details and attribution.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

tomhu

Model Tree

Base

Qwen/Qwen3-Coder-Next-FP8

Quantized

this model

Input Modalities

Text

Output Modalities