GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Avesed
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Avesed
Model Tree
Base
Qwen/Qwen3.6-27B
Quantized
this model
Input Modalities
Text
Image
Video
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
int4 weight bandwidth (fast decode) + int8 tensor-core compute (fast prefill) — the best serving quant on the NVIDIA Ampere line (A100 / RTX 3090).
vLLM gates its W4A8 kernels to Hopper. On Ampere the Marlin kernel can run W4A8-int8 but needs a small enablement patch — use vllm-ampere-optimized (prebuilt wheel + Docker image, or the standalone hot-patch). On Hopper it runs out of the box.
| concurrency | output tok/s | median TTFT | median TPOT |
|---|---|---|---|
| 1 (single-user) | 46.8 | 0.84 s | 19.8 ms |
| 32 (saturated) | 416 | 14.4 s | 63.6 ms |
Peak VRAM ~22.8 GiB/card. Single-user ~47 tok/s with sub-second TTFT; saturates ~416 tok/s aggregate.