GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AminFa12
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
AminFa12
Model Tree
Qwen/Qwen3-Coder-30B-A3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
evol-codealpaca and SWE-smith trajectories — so the surviving experts are the ones
this workload actually exercises.mlp.gate routers are excluded from quantization (kept in high precision — routers are
too sensitive for 4-bit).EvalPlus (greedy), measured on the pruned bf16 checkpoint.
| model | HumanEval | HumanEval+ | MBPP | MBPP+ |
|---|---|---|---|---|
| REAP-50 (this model, bf16 pre-quant) | 0.909 | 0.872 | 0.839 | 0.704 |
| REAP-40 (lighter prune, for reference) | 0.933 | 0.878 | 0.852 | 0.709 |
Requires a Blackwell GPU (compute capability 12.0) and vLLM ≥ 0.24 (older builds lack SM120 NVFP4 kernels).
bash
uv pip install -U vllmVLLM_ATTENTION_BACKEND=TRITON_ATTN \vllm serve AminFa12/qwen3-coder-30b-a3b-reap50-nvfp4 \ --max-model-len 32768 --gpu-memory-utilization 0.9 --enforce-eager \ --enable-auto-tool-choice --tool-call-parser qwen3_coderFirst-launch warmup: FlashInfer JIT-compiles its NVFP4 kernels for SM120 on the first run — this can look like a hang (GPU idle, one CPU core busy) for several minutes before the server binds. Be patient the first time; it caches afterwards.
--enforce-eagerskips the separate torch.compile/CUDA-graph phase.
python
from openai import OpenAIc = OpenAI(base_url="http://localhost:8000/v1", api_key="x")print(c.chat.completions.create( model="AminFa12/qwen3-coder-30b-a3b-reap50-nvfp4", messages=[{"role":"user","content":"Write a Python function to merge two sorted lists."}], temperature=0.7, top_p=0.8).choices[0].message.content)Licensed Apache-2.0, inherited from the Qwen3-Coder base model.
uv pip install -U vllmVLLM_ATTENTION_BACKEND=TRITON_ATTN \vllm serve AminFa12/qwen3-coder-30b-a3b-reap50-nvfp4 \ --max-model-len 32768 --gpu-memory-utilization 0.9 --enforce-eager \ --enable-auto-tool-choice --tool-call-parser qwen3_coderfrom openai import OpenAIc = OpenAI(base_url="http://localhost:8000/v1", api_key="x")print(c.chat.completions.create( model="AminFa12/qwen3-coder-30b-a3b-reap50-nvfp4", messages=[{"role":"user","content":"Write a Python function to merge two sorted lists."}], temperature=0.7, top_p=0.8).choices[0].message.content)