GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
dudeman2512
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
dudeman2512
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
| Variant | Format | Size | vs BF16 | Mean rel. error | Linears quantized | Left BF16 |
|---|---|---|---|---|---|---|
| Qwen3.8-27B-FP8 | float-quantized | 30.35 GB | 55% | 0.0264 | 615 | 0 |
| Qwen3.8-27B-NVFP4 | nvfp4-pack-quantized | 19.29 GB | 35% | 0.0950 | 615 | 0 |
| Qwen3.8-27B-int4 ← this one | pack-quantized | 18.31 GB | 33% | 0.1178 | 588 | 27 |
Mean relative error is ||dequant(W) - W|| / ||W||, averaged over a sample of quantized Linear layers, measured against the original BF16 weights. Lower is better.
| Format | pack-quantized |
| Weight bits | 4 |
| Group size | 128 |
| Strategy | group |
| Linears quantized | 588 |
| Left in BF16 | 27 |
| Shards | 5 |
| On disk | 18.31 GB |
27 Linear layers could not be quantized at group size 128: their in_features is not a multiple of 128. They are stored as BF16 and listed in the ignore list of config.json, so servers load them unquantized rather than misreading them.
They are all in the vision tower, so the language model is fully quantized:
markdown
model.visual.blocks.0.mlp.linear_fc2model.visual.blocks.1.mlp.linear_fc2model.visual.blocks.10.mlp.linear_fc2model.visual.blocks.11.mlp.linear_fc2model.visual.blocks.12.mlp.linear_fc2model.visual.blocks.13.mlp.linear_fc2... and 21 more
bash
vllm serve dudeman2512/Qwen3.8-27B-int4
Every produced tensor is checked for shape/dtype conformance against what the server expects, then reconstruction error is measured against the source BF16 weights, before anything is published. The numbers in the table above are those measurements — not estimates.
| Mean relative error | 0.1178 |
| Shape/dtype conformance failures | 0 |