GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
alanjcs
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
alanjcs
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
Claw-Eval thinking tokens are per-task (agentic; not a single-turn trace).
Settings
Models: base Qwen/Qwen3.6-27B vs bottlecapai/ThinkingCap-Qwen3.6-27B (shown as Ours in the table).
Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
Decoding: thinking on; sampling temperature=1.0, top_p=0.95, top_k=20, min_p=0.0 (bottlecapai/ThinkingCap-Qwen3.6-27B uses the base's sampling).
Max generation tokens: 100,000 for the general suite (gpqa_diamond, mmlu_pro, longbench_v2, realworldqa) and AA-LCR; 250,000 for HMMT (Nov 2025); 32,768 for supergpqa and livecodebench; 16,384 for ceval and mmlu_redux; 15,000 for llm-system-prompts-benchmark; 49,152 for Claw-Eval.
Metrics — the columns mirror the table:
<think> trace (think-per-task for Claw-Eval).Ours are paired on the same question (each side seed-averaged), each question's (base − cap)/base is taken, then averaged over shared questions (a larger ↓ = a bigger saving).We separately track two trace-quality failure modes, reported only in aggregate: looping — the model gets stuck repeating the same reasoning chain (sometimes a single sentence), never finishing its thinking; detected from the fraction of repetitive n-grams — and truncation — the <think> trace never closes because the model hits the generation-token cap while still reasoning, so no answer is produced. Across all out-of-domain responses, truncation drops from 2.9% to 0.4% while looping stays negligible (~0.2%).
Holdout test splits of datasets whose train splits are part of the finetuning mix — quality retention on in-distribution tasks (in contrast to the out-of-domain benchmarks above).
Settings
temperature=1.0, top_p=0.95, top_k=20, min_p=0.0 (bottlecapai/ThinkingCap-Qwen3.6-27B uses the base's sampling).Brevity finetuning leaves safety behaviour intact: on both safety sets bottlecapai/ThinkingCap-Qwen3.6-27B refuses harmful/jailbreak prompts at the base model's rate (statistically indistinguishable) while still spending fewer thinking tokens.
Settings
Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
Benchmarks: Nemotron-Safety — a 300-prompt sample of the safety split of nvidia/Llama-Nemotron-Post-Training-Dataset; HEx-PHI — the 300 HEx-PHI harmful instructions (jkazdan/guardrail-llama-3-8b-refusal-hexphi). Both are safety-relevant / jailbreak prompts.
Both datasets only contain a train subset and given their near-perfect scores, it is hard to rule out possible contimination in Qwen's original training data. These results are intended purely as a demonstration of preserved guardrail capabilities, and are in line with our internal usage testing.
Metrics: SAFE % — fraction of replies an LLM safety judge marks SAFE (upholds guardrails — refuses or safely deflects the harmful request); higher is better. Thinking tokens and Reduction are as defined for the token-efficiency table above.
Don't want to run the weights yourself? ThinkingCap-Qwen3.6-27B is hosted and served over an API on Requesty — call it directly, no local GPU needed.
python
from transformers import AutoModelForImageTextToText, AutoProcessormodel = AutoModelForImageTextToText.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B", dtype="bfloat16")proc = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B")
Check https://huggingface.co/Qwen/Qwen3.6-27B for recommended usage, sampling params etc.
Serve the bf16 model with either engine — plain, or with the model's own MTP (multi-token-prediction / NextN) head for self-speculative decoding (no separate draft model needed):
bash
# vLLM — standardvllm serve bottlecapai/ThinkingCap-Qwen3.6-27B# vLLM — with MTP self-speculative decoding (vLLM ≥ 0.24.0)vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B --speculative-config '{"method":"mtp","num_speculative_tokens":3}'# SGLang — standardpython -m sglang.launch_server --model-path bottlecapai/ThinkingCap-Qwen3.6-27B --trust-remote-code# SGLang — with MTP self-speculative decodingpython -m sglang.launch_server --model-path bottlecapai/ThinkingCap-Qwen3.6-27B --trust-remote-code \--speculative-algorithm EAGLE --speculative-num-steps 3 \--speculative-eagle-topk 1 --speculative-num-draft-tokens 4
MTP speculative decoding is lossless — the output is identical to standard decoding — and on the bf16 weights it accepts ≈3.34 drafts per verify step, a further ≈2.6× decode throughput on top of the finetune's token savings.
For GPU serving at half the memory, an official FP8 build lives at bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 — near-lossless, loaded natively by vLLM (vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8), with the MTP head kept in bf16; full serving benchmarks are on that card.
Quantized GGUF builds of this model live in the sibling repo bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF, for local inference with llama.cpp and compatible runtimes (Ollama, LM Studio, …).
Quantization stores the weights at reduced precision — e.g. ~4.7 bits per weight for Q4_K_M instead of 16-bit bf16 — cutting download size and memory severalfold at a small quality cost. Q4_K_M is the recommended size/quality balance, Q8_0 is near-lossless.
bash
llama-cli -hf bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M -p "Hi"
If you use this model, please cite:
bibtex
@misc{ThinkingCap-Qwen3.6-27B,title = {bottlecapai/ThinkingCap-Qwen3.6-27B},author = {Lasocki, Karol and Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Mikolov, Tomas},year = {2026},}
We acknowledge EuroHPC Joint Undertaking for awarding the project ID EHPC-AIF-2025SC03-029 access to Leonardo at CINECA, Italy.