GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
berkerdooo
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
berkerdooo
Model Tree
orcarouter/Qwen3.8-27B-Uncensored
Input Modalities
Output Modalities
Supported Functionality
AutoRound main @ b9f3d0079d014c73a1ff009800c597b9bc3f2a36 (version string 0.15.0), transformers 5.15.1, torch 2.13.0+cu130, one RTX PRO 6000 Blackwell.
scheme="W8A16" (bits 8, group_size 128, sym), iters=500, nsamples=1024, seqlen=2048, batch_size=4, gradient_accumulate_steps=2, low_gpu_mem_usage=False, seed=42.
Calibration: 256 samples built from NeelNanda/pile-10k + 768 from codeparrot/github-code-clean (documents concatenated so every sample is >= 2048 tokens, then truncated to 2048).
Every layer is named in full in layer_config (avoids AutoRound's shared-dict regex aliasing bug). Tuning took 1.26 h.
Teacher-forced top-24 logprobs on one 128,000-token wikitext-103 stream (rows 100k+ of the train split), one sequence, BF16 KV cache, vLLM 0.27.1, KL(P_bf16 || Q_int8) in nats over the truncated top-24.
These numbers are only comparable to other models scored with the same script, stream and teacher.
| depth | n | KL mean | KL p50 | KL p99 | top-1 agreement | ΔNLL |
|---|---|---|---|---|---|---|
| 0k-4k | 3,999 | 0.00189 | 0.00056 | 0.0239 | 97.67% | +0.0038 |
| 4k-16k | 12,000 | 0.00363 | 0.00078 | 0.0338 | 97.51% | +0.0011 |
Own NLL: BF16 1.8244, INT8 1.8265. For reference, the same script on Qwen/Qwen3.8-27B gives FP8 (Qwen/Qwen3.8-27B-FP8) KL 0.0048 / top-1 96.5% and Minachist's INT8 0.0029 / 97.2%.
bash
vllm serve <this-repo> --tensor-parallel-size 2 --max-model-len 262144 --trust-remote-code
Tested with vLLM 0.27.1 (Using MarlinLinearKernel for AutoGPTQLinearMethod). MTP speculative decoding: --speculative-config '{"method":"mtp","num_speculative_tokens":3}'.
| 16k-48k | 32,000 | 0.00264 | 0.00085 | 0.0293 | 97.22% | +0.0019 |
| 48k-128k | 80,000 | 0.00320 | 0.00088 | 0.0338 | 97.28% | +0.0022 |