GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Avesed
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Avesed
Model Tree
Avesed/Qwopus3.6-27B-v2-abliterated
Input Modalities
Output Modalities
Supported Functionality
int4 weight bandwidth (fast decode) + int8 compute (fast prefill). Measured on 2×RTX 3090 (cudagraph):
| single-stream decode | batch-16 decode | prefill | |
|---|---|---|---|
| W4A16 (int4) | 50.6 | 387 | 1045 tok/s |
| W8A8 (int8) | 38.8 | 343 | 1250 tok/s |
| W4A8 (this) | 50.4 | 393 | 1229 tok/s |
= W4A16's decode + W8A8's prefill, and small enough to keep CUDA graphs.
On Hopper it runs out of the box (Cutlass/Machete W4A8 kernels). On Ampere, vLLM's W4A8-int Marlin path is gated by a small config bug (vllm#38064); the #38066 fix (3 Python files, no recompile) enables the Ampere Marlin W4A8-int8 kernel — packaged at vllm-ampere-optimized.