GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
0xSero
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
0xSero
Model Tree
0xSero/GLM-4.7-185B
Input Modalities
Output Modalities
Supported Functionality
| Base model | 0xSero/GLM-4.7-185B |
| Format | W4A16 |
| Total params | 185B |
| Active / token | — |
| Experts / layer | 80 |
| Layers | 92 |
| Hidden size | 5120 |
| Context | 202,752 |
| On-disk size | 99 GB |
| Variant | Format | Link |
|---|---|---|
GLM-4.7-185B | BF16 | link |
GLM-4.7-185B-W4A16 (this) | W4A16 | link |
GLM-4.7-202B | BF16 | link |
50% Expert-Pruned + INT4 Quantized — Double compression for efficient deployment.
| Property | Value |
|---|---|
| Base Model | GLM-4.7-REAP-50 |
| Original (GLM-4.7) | 358B params, ~700GB |
| After REAP 50% | 179B params |
| After W4A16 Quant | ~92GB on disk |
| Quantization | INT4 weights, FP16 activations |
| Group Size | 128 |
| Format | GPTQ (AutoRound) |
text
GLM-4.7 (358B, 700GB) │ ▼ REAP 50% expert pruning │GLM-4.7-REAP-50 (179B) │ ▼ AutoRound W4A16 quantization │GLM-4.7-REAP-50-W4A16 (~92GB) ◀── This model Total: ~6.5x compressionREAP's effectiveness depends critically on calibration data that represents the target use case. We specifically optimized for code generation, function/tool calling, and agentic workflows.
| Dataset | Samples | Purpose | Why It Matters |
|---|---|---|---|
| evol-codealpaca-v1 | 700 | Code generation | 51% of mix — Code tasks activate specific expert pathways; pruning without code calibration destroys coding ability |
| xlam-function-calling-60k | 330 | Function/tool calling | 24% of mix — Tool use requires structured JSON output; experts handling schema generation must be preserved |
| SWE-smith-trajectories |
text
REAP Algorithm:1. Forward pass calibration samples through model2. Record which experts activate and their magnitudes3. Compute saliency = router_weight × activation_norm4. Prune lowest-saliency experts Key Insight: Experts are TASK-SPECIFIC├── Some experts specialize in natural language├── Some experts specialize in code syntax├── Some experts specialize in JSON/structured output└── Some experts specialize in multi-turn context If calibration lacks code → code-specialized experts appear "unused" → get pruned → model loses coding abilityCerebras used the same 3 datasets in their GLM-4.6 REAP experiments:
We followed this exact recipe for reproducibility.
Our calibration mix: 0xSero/glm47-reap-calibration-v2
bash
vllm serve 0xSero/GLM-4.7-185B-W4A16 \ --tensor-parallel-size 4 \ --trust-remote-code \ --quantization gptqpython
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "0xSero/GLM-4.7-185B-W4A16", device_map="auto", trust_remote_code=True)tokenizer = AutoTokenizer.from_pretrained("0xSero/GLM-4.7-185B-W4A16", trust_remote_code=True)python
#!/usr/bin/env python3"""REAP Pruning Script for MoE ModelsAdapted from: https://github.com/CerebrasResearch/reap""" import subprocessimport sys def run_reap( model_path: str, compression_ratio: float, dataset: str = "0xSero/glm47-reap-calibration-v2", samples: int = 1360, seed: int = 42, distance: str = "angular", reuse_observations: str = None,): """ Run REAP expert pruning. Args: model_path: Path to base model compression_ratio: 0.30 = prune 30%, keep 70% dataset: Calibration dataset (code + tools + agentic) samples: Number of calibration samples seed: Random seed for reproducibility distance: Distance metric for expert clustering reuse_observations: Path to pre-computed observations for instant pruning """ cmd = [ sys.executable, "src/reap/prune.py", "--model-name", model_path, "--dataset-name", dataset, "--compression-ratio", str(compression_ratio), "--prune-method", "reap", "--seed", str(seed), "--samples_per_category", str(samples), "--model_max_length", "2048", "--distance_measure", distance, "--record_pruning_metrics_only", "true", ] if reuse_observations: # Instant pruning: skip calibration, reuse precomputed expert scores cmd.extend(["--load_observations", reuse_observations]) subprocess.run(cmd, check=True) # Example: Create 40% pruned modelrun_reap( model_path="/path/to/GLM-4.7", compression_ratio=0.40, # Prune 40% of experts)python
#!/usr/bin/env python3"""AutoRound W4A16 QuantizationIntel's state-of-the-art weight quantization using signed gradient descent.""" from auto_round import AutoRound def quantize_w4a16( model_path: str, output_dir: str, bits: int = 4, group_size: int = 128, format: str = "auto_gptq",): """ Quantize model to INT4 weights with FP16 activations. Args: model_path: Path to REAP-pruned model output_dir: Output directory bits: Weight bit width (4 for W4A16) group_size: Quantization group size (128 is optimal) format: Output format (auto_gptq for vLLM compatibility) """ ar = AutoRound( model_path, scheme="W4A16", device="cuda", device_map="auto", trust_remote_code=True, batch_size=1, seqlen=512, nsamples=64, ) ar.quantize_and_save(output_dir, format=format) # Example: Quantize REAP-40 to W4A16quantize_w4a16( model_path="./GLM-4.7-REAP-40", output_dir="./GLM-4.7-REAP-40-W4A16",)Apache 2.0
License inherited from the base model.
bibtex
@misc{lasby2025reap, title = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression}, author = {Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa}, year = {2025}, eprint = {2510.13999}, archivePrefix = {arXiv}}Made possible by NVIDIA · TNG Technology · Lambda · Prime Intellect · Hot Aisle.
GLM-4.7 (358B, 700GB) │ ▼ REAP 50% expert pruning │GLM-4.7-REAP-50 (179B) │ ▼ AutoRound W4A16 quantization │GLM-4.7-REAP-50-W4A16 (~92GB) ◀── This modelTotal: ~6.5x compressionREAP Algorithm:1. Forward pass calibration samples through model2. Record which experts activate and their magnitudes3. Compute saliency = router_weight × activation_norm4. Prune lowest-saliency expertsKey Insight: Experts are TASK-SPECIFIC├── Some experts specialize in natural language├── Some experts specialize in code syntax├── Some experts specialize in JSON/structured output└── Some experts specialize in multi-turn contextIf calibration lacks code → code-specialized experts appear "unused" → get pruned → model loses coding abilityvllm serve 0xSero/GLM-4.7-185B-W4A16 \ --tensor-parallel-size 4 \ --trust-remote-code \ --quantization gptq@misc{lasby2025reap, title = {REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression}, author = {Mike Lasby and Ivan Lazarevich and Nish Sinnadurai and Sean Lie and Yani Ioannou and Vithursan Thangarasa}, year = {2025}, eprint = {2510.13999}, archivePrefix = {arXiv}}GLM-4.7-218B-W4A16 | W4A16 | link |
GLM-4.7-REAP-40-W4A16 | W4A16 | link |
| 80 (was 160) |
| VRAM Required | ~100GB |
| 330 |
| Agentic multi-turn |
| 24% of mix — Real SWE-bench trajectories with tool calls, file edits, and multi-step reasoning |