Requires transformers >= 5.13.0 (the qwen3_5_moe architecture). The model is multimodal, so it loads
via AutoModelForImageTextToText; text-only generation works exactly as with any chat model.
import torch
from transformers import AutoTokenizer, AutoModelForImageTextToText
repo = "KIEFERSA/Qwen3.6-35B-A3B-greek-reap25"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
repo, torch_dtype="auto", device_map="auto", trust_remote_code=True).eval()
msgs = [{"role": "user", "content": "Εξήγησε με απλά λόγια τι είναι η φωτοσύνθεση."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
Serving with vLLM
Fits on a single 80 GB+ GPU (~54 GB weights); use tensor parallelism for throughput on multi-GPU nodes.
Requires a vLLM build with qwen3_5_moe support.
# single GPU
vllm serve KIEFERSA/Qwen3.6-35B-A3B-greek-reap25 \
--trust-remote-code \
--dtype bfloat16 \
--max-model-len 8192 \
--gpu-memory-utilization 0.90 \
--served-model-name qwen3.6-greek-reap25
# 2x GPU (e.g. 2x B200) for higher throughput
vllm serve KIEFERSA/Qwen3.6-35B-A3B-greek-reap25 \
--trust-remote-code --dtype bfloat16 \
--tensor-parallel-size 2 --max-model-len 8192 \
--gpu-memory-utilization 0.90 --served-model-name qwen3.6-greek-reap25
Then query the OpenAI-compatible endpoint:
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "qwen3.6-greek-reap25",
"messages": [{"role": "user", "content": "Γράψε ένα σύντομο ποίημα για τη θάλασσα."}]
}'
Evaluations
All models below were evaluated with an identical harness: first-token letter-logit scoring for
knowledge MCQ (ARC, Belebele, MMLU, GreekMMLU) and length-normalized log-likelihood for the
no-natural-letter benchmarks (HellaSwag, Winogrande, TruthfulQA, Medical). dense = the unpruned
256-expert base; reap-0.25/0.38/0.50 keep 192 / 160 / 128 experts. This repo is reap-0.25.
Greek NLU (accuracy)
Table with columns: benchmark, dense, reap-0.25, reap-0.38, reap-0.50, Δ(0.25), Δ(0.50)| benchmark | dense | reap-0.25 | reap-0.38 | reap-0.50 | Δ(0.25) | Δ(0.50) |
|---|
| arc_challenge | 0.9144 | 0.9178 | 0.9170 | 0.9058 | +0.0034 | -0.0086 |
| arc_easy | 0.9672 | 0.9668 | 0.9613 | 0.9554 | -0.0004 |
English NLU (accuracy) — cross-lingual forgetting check
Table with columns: benchmark, dense, reap-0.25, reap-0.38, reap-0.50, Δ(0.25), Δ(0.50)| benchmark | dense | reap-0.25 | reap-0.38 | reap-0.50 | Δ(0.25) | Δ(0.50) |
|---|
| arc_challenge | 0.9599 | 0.9582 | 0.9590 | 0.9497 | -0.0017 | -0.0102 |
| arc_easy | 0.9912 | 0.9882 | 0.9886 | 0.9848 | -0.0030 |
Greek held-out perplexity (lower is better)
Table with columns: dense, reap-0.25, reap-0.38, reap-0.50| dense | reap-0.25 | reap-0.38 | reap-0.50 |
|---|
| 11.59 | 10.17 | 8.45 | 7.93 |
Note on perplexity vs. capability. Perplexity improves monotonically as more experts are pruned,
yet task accuracy degrades monotonically. Pruning sharpens the token distribution (lower NLL) while
removing stored knowledge — so perplexity is the wrong signal for choosing a compression ratio here.
Damage is concentrated in MMLU-style factual knowledge; grammar, commonsense and NLI benchmarks are
essentially unaffected at 0.25. Full per-benchmark JSON and the report are under evaluation/ in this repo.
Citation
@misc{kirouane2026qwen36greekreap,
title = {Qwen3.6-35B-A3B Greek-calibrated REAP (25% expert pruning)},
author = {Kirouane, Ayoub},
year = {2026},
howpublished = {Hugging Face, KIEFERSA/Qwen3.6-35B-A3B-greek-reap25}
}
REAP method:
@misc{reap2025,
title = {REAP: Router-weighted Expert Activation Pruning for Mixture-of-Experts},
author = {Cerebras Research},
howpublished = {\url{https://github.com/CerebrasResearch/reap}}
}