GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
prism-ml
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
prism-ml
Model Tree
Input Modalities
Output Modalities
Supported Functionality
bash
pip install sglangpython -m sglang.launch_server \--model prism-ml/Ternary-Bonsai-27B-AWQ-4bit \--port 8000 \--dtype bfloat16
bash
curl http://localhost:8000/v1/chat/completions \-H "Content-Type: application/json" \-d '{"model":"prism-ml/Ternary-Bonsai-27B-AWQ-4bit","messages":[{"role":"user","content":"Who are you?"}],"max_tokens":100}'
bash
# TP=2 DP=4 — 4 replicas, each split across 2 GPUs (throughput sweet spot)python -m sglang.launch_server \--model prism-ml/Ternary-Bonsai-27B-AWQ-4bit \--tp-size 2 --dp-size 4 \--load-balance-method total_tokens \--port 8000 --dtype bfloat16# TP=2 — single replica split across 2 GPUspython -m sglang.launch_server \--model prism-ml/Ternary-Bonsai-27B-AWQ-4bit \--tp-size 2 \--port 8000 --dtype bfloat16