GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
protogeni
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
protogeni
Model Tree
noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound
Input Modalities
Output Modalities
Supported Functionality
Use the patched syv-ai runtime, tested with vLLM 0.27.1. 100K profile details.
Download into the runtime repository:
bash
hf download protogeni/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4 \ --local-dir models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4Merge these settings into its .env:
dotenv
MODEL=/app/models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4CTX=fastSPEC=mtpDRAFT_TOKENS=4PREFIX_CACHE=1VISION=0GPU_UTIL=0.95MAX_LEN=100000MAX_SEQS=4CG=8EXTRA_ARGS=--kv-cache-memory=7730941132 --max-num-batched-tokens 512bash
docker compose --profile single up -d --no-deps --force-recreate singleSet the client's context limit to 100,000 tokens, including the reply.
RTX 3090 Ti, BF16 KV + MTP4. Generation rates exclude prompt processing and include reasoning tokens.
| Workload | Generation speed |
|---|---|
| Three short coding prompts, pooled | 128.4 tokens/s |
| 64,230 input + 1,024 output tokens | 92.1 tokens/s |
| 96,238 input + 3,072 output tokens | 82.7 tokens/s |
| 98,233 input + 1,536 output tokens | 84.6 tokens/s |
| Benchmark | Result |
|---|---|
| GSM8K | 94.77% — 1,250/1,319 |
| MMLU-Pro, 5-shot | 76.14% — 1,066/1,400 |
| HumanEval | 94.51% pass@1 — 155/164 |
| HumanEval+ | 90.85% pass@1 — 149/164 |
| IFBench | 39.33% strict / 42.00% loose |
| MMMU validation | 24.56% — 221/900; 891 output-cap hits |
MMLU-Pro used thinking; the other listed tests used thinking off. MMMU used an eager, MTP-off profile. Protocols, raw results, and earlier speed tests.
Qwen/Qwen3.8-27B → OrcaRouter uncensored → noon-at-cgn AutoRound W4A16 → this serving-optimized checkpoint.
hf download protogeni/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4 \ --local-dir models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4docker compose --profile single up -d --no-deps --force-recreate singleMODEL=/app/models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4CTX=fastSPEC=mtpDRAFT_TOKENS=4PREFIX_CACHE=1VISION=0GPU_UTIL=0.95MAX_LEN=100000MAX_SEQS=4CG=8EXTRA_ARGS=--kv-cache-memory=7730941132 --max-num-batched-tokens 512