GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Shaaf-Salman
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Shaaf-Salman
Model Tree
Qwen/Qwen3.5-122B-A10B
Input Modalities
Output Modalities
Supported Functionality
The base model is a sparse MoE with 256 routed experts × 48 layers (plus one always-on shared expert per layer). REAP scores every routed expert by its contribution and removes the least useful:
markdown
saliency(expert e) = mean over tokens routed to e of ( ||expert_e(x)||₂ · w_norm[t,e] )
where w_norm is the renormalized top-k router weight. The 94 lowest-saliency experts per layer
were pruned, keeping 162/256. The router, shared expert, attention, and all other components are
retained; num_experts_per_tok stays 8, so the model remains ~10B active per token — pruning
reduces total/memory footprint, not per-token compute.
| Base | This model | |
|---|---|---|
| Total params | ~122.6 B | ~79.97 B |
| Routed experts / layer | 256 | 162 (94 pruned) |
| Active experts / token | 8 routed + 1 shared | 8 routed + 1 shared |
| Precision | bf16 | bf16 |
Medical calibration mix (chat-templated, ~2.1M tokens), blended:
FreedomIntelligence/medical-o1-reasoning-SFT — clinical reasoning (75%)lavita/ChatDoctor-HealthCareMagic-100k — patient/doctor Q&A (25%)Experts important for the medical distribution are preserved; experts that rarely fire on medical text are the ones pruned.
The architecture is natively supported by vLLM (≥0.22). On 2×80GB GPUs, serve text-only in bf16:
bash
vllm serve <path> \--tensor-parallel-size 2 --dtype bfloat16 \--language-model-only --enforce-eager \--max-model-len 4096 --gpu-memory-utilization 0.97 \--trust-remote-code
Qwen3.5 reasons by default; disable per request with
chat_template_kwargs: {"enable_thinking": false}.
bash
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"<name>","messages":[{"role":"user","content":"First-line management of DKA? Answer concisely."}],"max_tokens":256, "temperature":0.7, "top_p":0.8, "top_k":20,"chat_template_kwargs":{"enable_thinking":false}}'
The native context length is 262,144 tokens; the example above caps it at 4096 to fit a small KV cache in bf16 on 2 GPUs. For larger context or higher concurrency, use more GPUs or FP8.
Apache-2.0, inherited from the base model Qwen/Qwen3.5-122B-A10B. Method: REAP
(Cerebras Research).