IAmTheRealGumbo

Qwen3.6-35B-A3B-REAP-192-experts

README

License: apache-2.0

Results

Table with columns: Model, Experts per layer, Experts kept, Total params, Size (bf16), vLLM GPU memory (weights), HumanEval pass@1
Model	Experts per layer	Experts kept	Total params	Size (bf16)	vLLM GPU memory (weights)	HumanEval pass@1
Qwen/Qwen3.6-35B-A3B (unpruned)	256	100%	35.95B	71.9 GB	64.7 GiB	94.5% (155/164)
Qwen3.6-35B-A3B-REAP-231-experts	231	90%	31.96B	63.9 GB	58.8 GiB	95.1% (156/164)
Qwen3.6-35B-A3B-REAP-192-experts	192	75%	27.05B	54.1 GB	49.7 GiB	93.3% (153/164)
Qwen3.6-35B-A3B-REAP-128-experts	128	50%	18.99B	38.0 GB	34.7 GiB	94.5% (155/164)
Qwen3.6-35B-A3B-REAP-64-experts	64	25%	10.93B	21.9 GB	19.7 GiB	85.4% (140/164)
Qwen3.6-35B-A3B-REAP-26-experts	26	10%	6.15B	12.3 GB	10.7 GiB	9.1% (15/164)
Qwen3.6-35B-A3B-REAP-8-experts	8	3%	3.88B	7.8 GB	6.5 GiB	0.0% (0/164)

HumanEval: all 164 problems, pass@1, greedy decoding (temperature 0), thinking disabled (enable_thinking: false), max 1024 new tokens. Each model was served with vLLM 0.19 (NVIDIA vllm:26.04-py3 container) on an NVIDIA DGX Spark (GB10), prompted through the chat template to return the complete function in a code block, and scored with openai/human-eval.
pass@1 is the fraction of problems whose single generated answer passes all of the problem's unit tests (one greedy sample per problem, no retries).
Noise: two identical runs of the unpruned model scored 93.9% and 94.5%, so differences of 1–2 problems (about 1 point) are within run-to-run noise. In particular the 231-, 192- and 128-expert variants are not distinguishable from the unpruned model on this benchmark.
vLLM GPU memory is the weight memory vLLM reports at load ("Model loading took ..."), in GiB. Total usage on top of that depends on the KV cache size you configure.
Size is the total size of the safetensors files (decimal GB, as shown by Hugging Face). Parameter counts include the 0.45B vision encoder.

How it was pruned

Method: REAP scores each expert by the average of its router gate weight times the L2 norm of its output, over the tokens routed to it. The lowest-scoring experts in each layer are removed, along with their router rows. Top-k router weights are renormalized when scoring, matching the model's own routing.
Calibration data: 1024 samples from theblackcat102/evol-codealpaca-v1 (code instructions), up to 2048 tokens each, seed 42. All variants come from the same calibration pass; only the number of removed experts differs.
Unchanged: attention and linear-attention layers, the shared expert in every MoE layer, the vision encoder, the tokenizer and the chat template.
Removed: the multi-token-prediction (MTP) head. REAP does not calibrate it, so MTP speculative decoding is not available with this checkpoint.
No fine-tuning was done after pruning.
Code: CerebrasResearch/reap, with local changes to support the Qwen3.5/3.6 MoE architecture (stacked expert tensors). reap_args.yaml in this repo records the exact pruning arguments.

Usage

Same as the base model; the expert count is read from config.json.

bash

vllm serve IAmTheRealGumbo/Qwen3.6-35B-A3B-REAP-192-experts

With transformers (version 5 or later, which includes the qwen3_5_moe architecture):

python

from transformers import AutoModelForImageTextToText, AutoProcessor model_id = "IAmTheRealGumbo/Qwen3.6-35B-A3B-REAP-192-experts"model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")processor = AutoProcessor.from_pretrained(model_id)

Limitations

Only HumanEval was evaluated. Calibration used code data only, so expert selection favors coding; other domains (math, multilingual, general chat, agentic tool use) may degrade more than HumanEval suggests.
The vision encoder is unchanged, but image and video understanding were not evaluated after pruning.
HumanEval is small (164 problems) and saturated for this model family; it shows large regressions, like those of the 64-expert and smaller variants, but not small ones.

Credits

Base model by the Qwen team (Qwen/Qwen3.6-35B-A3B, Apache-2.0). REAP by Lasby et al., REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression (ICLR 2026).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

IAmTheRealGumbo

Model Tree

Base

Qwen/Qwen3.6-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality