KIEFERSA

Qwen3.6-35B-A3B-greek-reap25

README

License: apache-2.0

Usage (Transformers)

Requires transformers >= 5.13.0 (the qwen3_5_moe architecture). The model is multimodal, so it loads via AutoModelForImageTextToText; text-only generation works exactly as with any chat model.

python

import torchfrom transformers import AutoTokenizer, AutoModelForImageTextToText repo = "KIEFERSA/Qwen3.6-35B-A3B-greek-reap25"tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)model = AutoModelForImageTextToText.from_pretrained(    repo, torch_dtype="auto", device_map="auto", trust_remote_code=True).eval() msgs = [{"role": "user", "content": "Εξήγησε με απλά λόγια τι είναι η φωτοσύνθεση."}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=512, do_sample=False)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Serving with vLLM

Fits on a single 80 GB+ GPU (~54 GB weights); use tensor parallelism for throughput on multi-GPU nodes. Requires a vLLM build with qwen3_5_moe support.

bash

# single GPUvllm serve KIEFERSA/Qwen3.6-35B-A3B-greek-reap25 \  --trust-remote-code \  --dtype bfloat16 \  --max-model-len 8192 \  --gpu-memory-utilization 0.90 \  --served-model-name qwen3.6-greek-reap25 # 2x GPU (e.g. 2x B200) for higher throughputvllm serve KIEFERSA/Qwen3.6-35B-A3B-greek-reap25 \  --trust-remote-code --dtype bfloat16 \  --tensor-parallel-size 2 --max-model-len 8192 \  --gpu-memory-utilization 0.90 --served-model-name qwen3.6-greek-reap25

Then query the OpenAI-compatible endpoint:

bash

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{  "model": "qwen3.6-greek-reap25",  "messages": [{"role": "user", "content": "Γράψε ένα σύντομο ποίημα για τη θάλασσα."}]}'

Evaluations

All models below were evaluated with an identical harness: first-token letter-logit scoring for knowledge MCQ (ARC, Belebele, MMLU, GreekMMLU) and length-normalized log-likelihood for the no-natural-letter benchmarks (HellaSwag, Winogrande, TruthfulQA, Medical). dense = the unpruned 256-expert base; reap-0.25/0.38/0.50 keep 192 / 160 / 128 experts. This repo is reap-0.25.

Greek NLU (accuracy)

Table with columns: benchmark, dense, reap-0.25, reap-0.38, reap-0.50, Δ(0.25), Δ(0.50)
benchmark	dense	reap-0.25	reap-0.38	reap-0.50	Δ(0.25)	Δ(0.50)
arc_challenge	0.9144	0.9178	0.9170	0.9058	+0.0034	-0.0086
arc_easy	0.9672	0.9668	0.9613	0.9554	-0.0004

English NLU (accuracy) — cross-lingual forgetting check

Table with columns: benchmark, dense, reap-0.25, reap-0.38, reap-0.50, Δ(0.25), Δ(0.50)
benchmark	dense	reap-0.25	reap-0.38	reap-0.50	Δ(0.25)	Δ(0.50)
arc_challenge	0.9599	0.9582	0.9590	0.9497	-0.0017	-0.0102
arc_easy	0.9912	0.9882	0.9886	0.9848	-0.0030

Greek held-out perplexity (lower is better)

Table with columns: dense, reap-0.25, reap-0.38, reap-0.50
dense	reap-0.25	reap-0.38	reap-0.50
11.59	10.17	8.45	7.93

Note on perplexity vs. capability. Perplexity improves monotonically as more experts are pruned, yet task accuracy degrades monotonically. Pruning sharpens the token distribution (lower NLL) while removing stored knowledge — so perplexity is the wrong signal for choosing a compression ratio here. Damage is concentrated in MMLU-style factual knowledge; grammar, commonsense and NLI benchmarks are essentially unaffected at 0.25. Full per-benchmark JSON and the report are under evaluation/ in this repo.

Citation

bibtex

@misc{kirouane2026qwen36greekreap,  title        = {Qwen3.6-35B-A3B Greek-calibrated REAP (25% expert pruning)},  author       = {Kirouane, Ayoub},  year         = {2026},  howpublished = {Hugging Face, KIEFERSA/Qwen3.6-35B-A3B-greek-reap25}}

REAP method:

bibtex

@misc{reap2025,  title        = {REAP: Router-weighted Expert Activation Pruning for Mixture-of-Experts},  author       = {Cerebras Research},  howpublished = {\url{https://github.com/CerebrasResearch/reap}}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

KIEFERSA

Model Tree

Base

Qwen/Qwen3.6-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Usage (Transformers)

Requires transformers >= 5.13.0 (the qwen3_5_moe architecture). The model is multimodal, so it loads via AutoModelForImageTextToText; text-only generation works exactly as with any chat model.

python

import torchfrom transformers import AutoTokenizer, AutoModelForImageTextToText repo = "KIEFERSA/Qwen3.6-35B-A3B-greek-reap25"tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)model = AutoModelForImageTextToText.from_pretrained(    repo, torch_dtype="auto", device_map="auto", trust_remote_code=True).eval() msgs = [{"role": "user", "content": "Εξήγησε με απλά λόγια τι είναι η φωτοσύνθεση."}]ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)out = model.generate(ids, max_new_tokens=512, do_sample=False)print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Serving with vLLM

Fits on a single 80 GB+ GPU (~54 GB weights); use tensor parallelism for throughput on multi-GPU nodes. Requires a vLLM build with qwen3_5_moe support.

bash

# single GPUvllm serve KIEFERSA/Qwen3.6-35B-A3B-greek-reap25 \  --trust-remote-code \  --dtype bfloat16 \  --max-model-len 8192 \  --gpu-memory-utilization 0.90 \  --served-model-name qwen3.6-greek-reap25 # 2x GPU (e.g. 2x B200) for higher throughputvllm serve KIEFERSA/Qwen3.6-35B-A3B-greek-reap25 \  --trust-remote-code --dtype bfloat16 \  --tensor-parallel-size 2 --max-model-len 8192 \  --gpu-memory-utilization 0.90 --served-model-name qwen3.6-greek-reap25

Then query the OpenAI-compatible endpoint:

bash

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{  "model": "qwen3.6-greek-reap25",  "messages": [{"role": "user", "content": "Γράψε ένα σύντομο ποίημα για τη θάλασσα."}]}'

Evaluations

Greek NLU (accuracy)

Table with columns: benchmark, dense, reap-0.25, reap-0.38, reap-0.50, Δ(0.25), Δ(0.50)
benchmark	dense	reap-0.25	reap-0.38	reap-0.50	Δ(0.25)	Δ(0.50)
arc_challenge	0.9144	0.9178	0.9170	0.9058	+0.0034	-0.0086
arc_easy	0.9672	0.9668	0.9613	0.9554	-0.0004

English NLU (accuracy) — cross-lingual forgetting check

Table with columns: benchmark, dense, reap-0.25, reap-0.38, reap-0.50, Δ(0.25), Δ(0.50)
benchmark	dense	reap-0.25	reap-0.38	reap-0.50	Δ(0.25)	Δ(0.50)
arc_challenge	0.9599	0.9582	0.9590	0.9497	-0.0017	-0.0102
arc_easy	0.9912	0.9882	0.9886	0.9848	-0.0030

Greek held-out perplexity (lower is better)

Table with columns: dense, reap-0.25, reap-0.38, reap-0.50
dense	reap-0.25	reap-0.38	reap-0.50
11.59	10.17	8.45	7.93

Note on perplexity vs. capability. Perplexity improves monotonically as more experts are pruned, yet task accuracy degrades monotonically. Pruning sharpens the token distribution (lower NLL) while removing stored knowledge — so perplexity is the wrong signal for choosing a compression ratio here. Damage is concentrated in MMLU-style factual knowledge; grammar, commonsense and NLI benchmarks are essentially unaffected at 0.25. Full per-benchmark JSON and the report are under evaluation/ in this repo.

Citation

bibtex

@misc{kirouane2026qwen36greekreap,  title        = {Qwen3.6-35B-A3B Greek-calibrated REAP (25% expert pruning)},  author       = {Kirouane, Ayoub},  year         = {2026},  howpublished = {Hugging Face, KIEFERSA/Qwen3.6-35B-A3B-greek-reap25}}

REAP method:

bibtex

@misc{reap2025,  title        = {REAP: Router-weighted Expert Activation Pruning for Mixture-of-Experts},  author       = {Cerebras Research},  howpublished = {\url{https://github.com/CerebrasResearch/reap}}}