GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
petra312
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
petra312
Model Tree
petra312/GLM-4.6-Derestricted-v3-NF4
Input Modalities
Output Modalities
Supported Functionality
Router-weighted Expert Activation Pruning (REAP) applied to petra312/GLM-4.6-Derestricted-v3-NF4.
lm_headtransformers==4.57.6, trust_remote_code=Truen_routed_experts=80, num_experts_per_tok=8. Load with the checkpoint's quantization_config (llm_int8_skip_modules=[] so lm_head stays Linear4bit).
python
from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "petra312/GLM-4.6-Derestricted-v3-NF4-REAP-50pct"tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", trust_remote_code=True, attn_implementation="sdpa",)Do not pass a second BitsAndBytesConfig; the repo already has one. Requires ~90 GiB disk and a mix of GPU + host RAM (experts do not fit in 32 GiB VRAM).