shawnw3i

Qwen3.8-27B-GPTQ-INT8-MTP

README

License: apache-2.0

Highlights

GPTQ Marlin kernel supported (auto-converted by vLLM at runtime)
MTP speculative decoding supported out of the box
80+ tok/s on a single A800 80GB (vLLM 0.27.1, MTP enabled, fp8 KV cache)
Vision inputs supported.

vLLM

markdown
vllm serve shawnw3i/Qwen3.8-27B-GPTQ-INT8-MTP \
  --max-model-len 65536 \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

shawnw3i

Model Tree

Base

Qwen/Qwen3.8-27B

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

Qwen3.8-27B-GPTQ-INT8-MTP API & Inference Endpoint | FriendliAI