GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Ar4ikov
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Ar4ikov
Model Tree
Ar4ikov/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM
Input Modalities
Output Modalities
Supported Functionality
The container does everything (download, verify, serve), with the vision tower on:
bash
git clone https://github.com/Ar4ikov/TurboQwen && cd TurboQwencp .env.example .env # CHECKPOINT=uncensored and MODEL=/app/models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fastdocker compose --profile single up -dBare metal, on the fork branch this was measured with (Ar4ikov/HyperQwen@awq-asym,
vLLM 0.29.0 + the series + marlin-int8-asym-zp):
bash
hf download Ar4ikov/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast --local-dir models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fastVISION=1 MODEL=$PWD/models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast SPEC=mtp CTX=fast bash single-user/start_qwen.shPlain vLLM 0.29 also loads it (vllm serve Ar4ikov/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast --max-model-len 65536), without the
speculative decoding, the draft head or the int8 Marlin path that need the patch series.
RTX 3090 (350 W), vLLM 0.29.0, HyperQwen bench/run_benchmarks.sh single, second run
after boot, VISION=1 (the tower streamed from host RAM per image). C1 = one stream of
real prompts with 1,024-token answers; tok/step = tokens accepted per forward pass.
| profile | C1 T=default | C1 T=0 | tok/step | C8 T=default | KV pool |
|---|---|---|---|---|---|
SPEC=mtp CTX=fast (64k, bf16 KV) | 112.7 tok/s | 127.6 | 2.74 / 2.95 | 434 tok/s | 80,185 |
SPEC=dflash2 CTX=fast KV_MEM=4600000000 DFLASH_MAX_LEN=49152 | 135.9 | 139.3 | 3.29 / 3.29 | 401 | 53,233 |
production line: DFlash2 k=15 + INT8_ACT=int8 PREFILL_ATTN=int8 (40k; TTFT 96 ms) | 116.7 | 135.5 | 2.97 / 3.44 | n/a (4 slots) | 42,113 |
Images are described correctly in every profile (a drawn red square, blue circle and a
line of text: boost/image_smoke.py). The whole table, the int8 (W4A8) profiles and the
kernel measurements: github.com/Ar4ikov/TurboQwen.
HyperQwen's pipeline rewrites the checkpoint in place (int8 heads, the draft head) and its launchers, verify script and Docker entrypoint expect that layout. Doing it once and publishing the result turns a ~10-minute CPU step per machine into a download, and keeps the original export as it is for transformers and plain vLLM users.
Quantization recipe of the body, calibration set and the AWQ mappings for the hybrid
Gated-DeltaNet / attention layers: see the source export's card and its recipe.yaml
(kept here). Root model: Qwen/Qwen3.8-27B, Apache-2.0.
git clone https://github.com/Ar4ikov/TurboQwen && cd TurboQwencp .env.example .env # CHECKPOINT=uncensored and MODEL=/app/models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fastdocker compose --profile single up -dhf download Ar4ikov/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast --local-dir models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fastVISION=1 MODEL=$PWD/models/Qwen3.8-27B-Uncensored-AWQ-W4A16-ASYM-HyperQwen-fast SPEC=mtp CTX=fast bash single-user/start_qwen.sh