AminFa12

qwen3-coder-30b-a3b-reap50-nvfp4

README

License: apache-2.0

How it was made

Prune — Cerebras REAP @ 50%. Each expert is scored by router-gate × activation-norm over a calibration set, and the 64 lowest-saliency experts per layer are removed, with router renormalization. The calibration set is personal: ~100 real coding-agent transcripts plus evol-codealpaca and SWE-smith trajectories — so the surviving experts are the ones this workload actually exercises.
Quantize — NVIDIA TensorRT Model-Optimizer 0.45, NVFP4. PTQ calibrated on the same personal transcripts (100 samples @ 8192 tokens). All 48 mlp.gate routers are excluded from quantization (kept in high precision — routers are too sensitive for 4-bit).

Evaluation

EvalPlus (greedy), measured on the pruned bf16 checkpoint.

Table with columns: model, HumanEval, HumanEval+, MBPP, MBPP+
model	HumanEval	HumanEval+	MBPP	MBPP+
REAP-50 (this model, bf16 pre-quant)	0.909	0.872	0.839	0.704
REAP-40 (lighter prune, for reference)	0.933	0.878	0.852	0.709

Usage (vLLM, SM120)

Requires a Blackwell GPU (compute capability 12.0) and vLLM ≥ 0.24 (older builds lack SM120 NVFP4 kernels).

bash
uv pip install -U vllm
VLLM_ATTENTION_BACKEND=TRITON_ATTN \
vllm serve AminFa12/qwen3-coder-30b-a3b-reap50-nvfp4 \
  --max-model-len 32768 --gpu-memory-utilization 0.9 --enforce-eager \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

First-launch warmup: FlashInfer JIT-compiles its NVFP4 kernels for SM120 on the first run — this can look like a hang (GPU idle, one CPU core busy) for several minutes before the server binds. Be patient the first time; it caches afterwards. --enforce-eager skips the separate torch.compile/CUDA-graph phase.

python
from openai import OpenAI
c = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
print(c.chat.completions.create(
    model="AminFa12/qwen3-coder-30b-a3b-reap50-nvfp4",
    messages=[{"role":"user","content":"Write a Python function to merge two sorted lists."}],
    temperature=0.7, top_p=0.8).choices[0].message.content)

Credits & license

Cerebras REAP — pruning method
Qwen3-Coder — base model (Apache-2.0)
NVIDIA TensorRT Model-Optimizer — NVFP4 PTQ

Licensed Apache-2.0, inherited from the Qwen3-Coder base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

AminFa12

Model Tree

Base

Qwen/Qwen3-Coder-30B-A3B-Instruct

Quantized

this model

Input Modalities

Text

Output Modalities