Load it
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "kueizen/Marco-Mini-Instruct-REAP30"tok = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
How this was made
- Pruning: REAP (Router-weighted Expert Activation Pruning, Cerebras Research), using the reference implementation. REAP scores each expert by its router weight times the size of its output over a calibration set, then removes the lowest-scoring experts whole. It is one-shot: no retraining. Seed 42.
- Calibration set: theblackcat102/evol-codealpaca-v1 (train split, shuffled, seed 42), 64 samples per category, batch size 1, max sequence length 2048 tokens.
- Experts removed: 30% (180 of 256 kept).
We have not run task benchmarks on these checkpoints (MMLU, coding, multilingual). Test them on your own workload before relying on them.
License and credits
Derived from ATH-MaaS/Marco-Mini-Instruct and released under the same Apache 2.0 licence. What we changed: removed experts with REAP. Nothing else was modified or retrained. REAP is by Cerebras Research: paper, code. All credit for the base model goes to its authors.
Published by Kueizen.