GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
sakamakismile
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
sakamakismile
Model Tree
AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16
Input Modalities
Output Modalities
Supported Functionality
| concurrency | aggregate t/s |
|---|---|
| 1 | 69.0 |
| 4 | 202.3 |
| 8 | 377.8 |
Single-stream prefill: 3,720 tok/s (best 3,888) on an 8k prompt, prefix cache disabled, mean of 3. GPU KV cache at this config: 619,613 tokens.
Sanity-checked after quantization: Japanese self-introduction, arithmetic/primality, and Python code generation all come back clean — no structural damage from the 4-bit pass.
bash
vllm serve sakamakismile/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4 \--trust-remote-code --tensor-parallel-size 4 \--max-model-len 32768 --kv-cache-dtype fp8 --reasoning-parser qwen3 \--speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'
On boards without P2P add NCCL_P2P_DISABLE=1 and --disable-custom-all-reduce.
Use reasoning_effort: "medium" for anything long-form. At the default (highest) effort this model can spend its entire token budget inside <think> — the reasoning block grows past 18,000 characters, sometimes degenerating into repeating a single line, and no answer is ever emitted. At medium effort the same prompts finish cleanly every time: I ran a 9-case gate (French and English long-form, temperature 0 and 0.7) in three configurations — NVFP4 with MTP num_speculative_tokens: 6, NVFP4 without MTP, and the original bf16 checkpoint with MTP — and all 27 runs passed with thinking held to 1–3k characters.
json
"chat_template_kwargs": {"reasoning_effort": "medium"}
Speculative decoding is fine — an earlier version of this card told you to disable it, which was wrong. MTP at n=6 passes the gate as long as the effort is medium. My apologies for the misdirection, and thanks to the user on Jetson / vLLM 0.27.1 who reported the looping.
The 15 mtp.* modules are listed in quantization_config.ignore — do not remove them. If vLLM treats the bf16 MTP head as NVFP4 the draft breaks silently: 0% acceptance, slower than no MTP at all.
W4A16 (NVFP4A16) does not serve on this architecture in vLLM 0.22 (gptq_marlin_repack: size_n=24 not divisible by tile_n_size=64). W4A4 only.
llm-compressor NVFP4 (W4A4, group 16), targets: [Linear], ignore: [lm_head, re:.*visual.*, re:.*conv1d.*, re:.*mtp.*], 32 calibration samples × 8192 tok from neuralmagic/calibration. MTP tensors grafted back in bf16 after saving and appended to quantization_config.ignore.
🙏 @AEON-7 for the model, Qwen team for the base, vLLM & llm-compressor teams for the tooling.
Long single-file code generation drops a closing paren roughly 1–2 times in 14 regardless of sampling temperature (measured with a JS parser on the base Qwen3.8-27B). Put a syntax check in the loop rather than tuning temperature.