GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
soyrsoyr
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
soyrsoyr
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
Linear layers only. The vision tower
(re:.*visual.*), the hybrid linear-attention mixers (re:.*linear_attn.*),
and lm_head stay in bf16. The full VLM (with vision_config) is saved in the
compressed-tensors format,
and the base MTP predictor is preserved for speculative decoding.python
from transformers import AutoModelForImageTextToText, AutoTokenizerfrom llmcompressor import oneshotfrom llmcompressor.modifiers.quantization import QuantizationModifierMODEL_ID = "Qwen/Qwen3.8-27B"model = AutoModelForImageTextToText.from_pretrained(MODEL_ID, dtype="bfloat16")tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)recipe = QuantizationModifier(targets="Linear",scheme="FP8_DYNAMIC",ignore=["lm_head", "re:.*visual.*", "re:.*linear_attn.*", "re:.*mlp.gate$"],)oneshot(model=model, recipe=recipe) # data-freemodel.save_pretrained("Qwen3.8-27B-FP8-dynamic", save_compressed=True, save_original_format=False)tokenizer.save_pretrained("Qwen3.8-27B-FP8-dynamic")
bash
vllm serve soyrsoyr/Qwen3.8-27B-FP8-dynamic
python
from vllm import LLM, SamplingParamsllm = LLM(model="soyrsoyr/Qwen3.8-27B-FP8-dynamic")out = llm.generate(["The capital of France is"], SamplingParams(max_tokens=32))print(out[0].outputs[0].text)
Recovery vs. the bf16 base, evaluated through the vLLM backend with lm-evaluation-harness (OpenLLM v1) and lighteval (generative reasoning, pass@1 at temperature 0.6, top_p 0.95, up to 32k tokens).
| Benchmark | Qwen3.8-27B | FP8-dynamic | Recovery |
|---|---|---|---|
| ARC-Challenge (25-shot), acc_norm | 50.68 | 49.91 | 98.5% |
| HellaSwag (10-shot), acc_norm | 71.99 | 72.38 | 100.5% |
| TruthfulQA-mc2 (0-shot), acc | 61.25 | 61.21 | 99.9% |
| Winogrande (5-shot), acc | 76.87 | 76.01 | 98.9% |
MMLU and GSM8K are omitted. Qwen3.8-27B is a reasoning model, so under the OpenLLM v1 protocol GSM8K has its chain of thought truncated and MMLU's loglikelihood is measured where the model wants to emit its think block, both of which collapse to a harness artifact rather than a real score. Math and knowledge are captured by the generative reasoning suite instead.
| Benchmark | Qwen3.8-27B | FP8-dynamic | Recovery |
|---|---|---|---|
| AIME-24, pass@1 | 90.00 | 90.00 | 100.0% |
| AIME-25, pass@1 | 93.33 | 90.00 | 96.4% |
| MATH-500, pass@1 | 82.00 | 82.20 | 100.2% |
| Average | 88.44 | 87.40 |
AIME-24 and AIME-25 are 30 problems each, scored at pass@1 with a single sample, so a swing of a few problems is within run-to-run sampling variance rather than a real capability change.
| Average | 65.20 | 64.88 | 99.5% |