At a glance

Which variant should I pick?

This repository now hosts the BF16 GLM-5 checkpoint produced by a 50% REAP prune. The actual checkpoint contents are the BF16 files described below.

Checkpoint

• Base model: GLM-5-BF16
• Architecture: GlmMoeDsaForCausalLM
• Method: refusal_contrast_reap
• Compression ratio: 0.50
• Seed: 42
• Router renormalization: true
• Parameters: 381,464,351,232
• Total safetensors size: 762,928,740,864 bytes
• Shards: 17
• Precision: BF16

Provenance

• Observation run: glm5-grouped-22k-20260331T172330Z
• Calibration dataset: combined
• Prune output directory: /data0/external_research/glm5-layerwise-reap-artifacts/GLM-5-BF16/combined/pruned_models/layerwise_refusal_contrast_reap-renorm_true-seed_42-0.50

Files

• model-00001-of-00017.safetensors through model-00017-of-00017.safetensors
• model.safetensors.index.json
• config.json
• generation_config.json
• chat_template.jinja
• tokenizer.json
• tokenizer_config.json
• reap_layerwise_args.yaml

Notes

• This upload replaces the older multi-shard checkpoint previously hosted in this repo.
• The metadata above reflects the actual checkpoint contents as of 2026-04-05.

License & citation

License inherited from the base model.

bibtex
Copy code
@misc{lasby2025reap,
  title  = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression},
  author = {Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa},
  year   = {2025}, eprint = {2510.13999}, archivePrefix = {arXiv}
}

Sponsors

Made possible by NVIDIA · TNG Technology · Lambda · Prime Intellect · Hot Aisle.