SushiRiceMan

ThinkingCap-Qwen3.6-27B-oQ5e-mtp

Deploy Dedicated

README

License: apache-2.0

Out-of-domain token efficiency

Claw-Eval thinking tokens are per-task (agentic; not a single-turn trace).

Settings

Models: base Qwen/Qwen3.6-27B vs bottlecapai/ThinkingCap-Qwen3.6-27B (shown as Ours in the table).
Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
Decoding: thinking on; sampling temperature=1.0, top_p=0.95, top_k=20, min_p=0.0 (bottlecapai/ThinkingCap-Qwen3.6-27B uses the base's sampling).
Max generation tokens: 100,000 for the general suite (gpqa_diamond, mmlu_pro, longbench_v2, realworldqa) and AA-LCR; 250,000 for HMMT (Nov 2025); 32,768 for supergpqa and livecodebench; 16,384 for ceval and mmlu_redux; 15,000 for llm-system-prompts-benchmark; 49,152 for Claw-Eval.
Metrics — the columns mirror the table:
- Accuracy (Base / Ours) — fraction correct (exact/regex match; soft compliance for llm-system-prompts-benchmark; judge task-score for Claw-Eval; judge CORRECT/INCORRECT for AA-LCR).
- Thinking tokens (Base / Ours) — mean length of the single-turn <think> trace (think-per-task for Claw-Eval).
- Reduction — the average per-question thinking-token saving: base and Ours are paired on the same question (each side seed-averaged), each question's (base − cap)/base is taken, then averaged over shared questions (a larger ↓ = a bigger saving).
- Macro average (bottom row) — equal-weight mean across benchmarks.
We separately track two trace-quality failure modes, reported only in aggregate: looping — the model gets stuck repeating the same reasoning chain (sometimes a single sentence), never finishing its thinking; detected from the fraction of repetitive n-grams — and truncation — the <think> trace never closes because the model hits the generation-token cap while still reasoning, so no answer is produced. Across all out-of-domain responses, truncation drops from 2.9% to 0.4% while looping stays negligible (~0.2%).

In-domain evals

Holdout test splits of datasets whose train splits are part of the finetuning mix — quality retention on in-distribution tasks (in contrast to the out-of-domain benchmarks above).

Settings

Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
Decoding: sampling temperature=1.0, top_p=0.95, top_k=20, min_p=0.0 (bottlecapai/ThinkingCap-Qwen3.6-27B uses the base's sampling).
Max generation tokens: 15,000 for GSM8K; 8,192 for the MCQ sets.
Data: GSM8K is the full 1,319-row test split; the MCQ sets are capped at 1,000 rows (OpenBookQA = 500 and QASC = 926 are smaller, so full).
Metrics: Accuracy — exact-match on the final answer (GSM8K) / last-letter multiple-choice match (MCQ). Thinking tokens, Reduction and Macro average are as defined for the token-efficiency table above, as are the looping and truncation failure modes: across all in-domain responses, truncation drops from 1.6% to 0.03% while looping is negligible for both (≤0.01%).

Guardrails preservation

Brevity finetuning leaves safety behaviour intact: on both safety sets bottlecapai/ThinkingCap-Qwen3.6-27B refuses harmful/jailbreak prompts at the base model's rate (statistically indistinguishable) while still spending fewer thinking tokens.

Settings

Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
Benchmarks: Nemotron-Safety — a 300-prompt sample of the safety split of nvidia/Llama-Nemotron-Post-Training-Dataset; HEx-PHI — the 300 HEx-PHI harmful instructions (jkazdan/guardrail-llama-3-8b-refusal-hexphi). Both are safety-relevant / jailbreak prompts.

Both datasets only contain a train subset and given their near-perfect scores, it is hard to rule out possible contimination in Qwen's original training data. These results are intended purely as a demonstration of preserved guardrail capabilities, and are in line with our internal usage testing.
Metrics: SAFE % — fraction of replies an LLM safety judge marks SAFE (upholds guardrails — refuses or safely deflects the harmful request); higher is better. Thinking tokens and Reduction are as defined for the token-efficiency table above.

Usage

HuggingFace Transformers

python
from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B", dtype="bfloat16")
proc = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B")

Check https://huggingface.co/Qwen/Qwen3.6-27B for recommended usage, sampling params etc.

FP8 (vLLM / SGLang)

An official FP8 quantization for GPU serving lives in the sibling repo bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 — half the memory of bf16 at near-lossless quality, in the compressed-tensors format vLLM/SGLang load natively, with the MTP (multi-token-prediction) speculative-decoding head kept in bf16.

bash
vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

GGUF (llama.cpp)

Quantized GGUF builds of this model live in the sibling repo bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF, for local inference with llama.cpp and compatible runtimes (Ollama, LM Studio, …).

Quantization stores the weights at reduced precision — e.g. ~4.7 bits per weight for Q4_K_M instead of 16-bit bf16 — cutting download size and memory severalfold at a small quality cost. Q4_K_M is the recommended size/quality balance, Q8_0 is near-lossless.

bash
llama-cli -hf bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M -p "Hi"

Where to find us

Citation

If you use this model, please cite:

bibtex
@misc{ThinkingCap-Qwen3.6-27B,
  title     = {bottlecapai/ThinkingCap-Qwen3.6-27B},
  author    = {Lasocki, Karol and Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Mikolov, Tomas},
  year      = {2026},
}

Acknowledgements

We acknowledge EuroHPC Joint Undertaking for awarding the project ID EHPC-AIF-2025SC03-029 access to Leonardo at CINECA, Italy.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

SushiRiceMan

Model Tree

Base

Qwen/Qwen3.6-27B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Out-of-domain token efficiency

Claw-Eval thinking tokens are per-task (agentic; not a single-turn trace).

Settings

Models: base Qwen/Qwen3.6-27B vs bottlecapai/ThinkingCap-Qwen3.6-27B (shown as Ours in the table).
Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
Decoding: thinking on; sampling temperature=1.0, top_p=0.95, top_k=20, min_p=0.0 (bottlecapai/ThinkingCap-Qwen3.6-27B uses the base's sampling).
Max generation tokens: 100,000 for the general suite (gpqa_diamond, mmlu_pro, longbench_v2, realworldqa) and AA-LCR; 250,000 for HMMT (Nov 2025); 32,768 for supergpqa and livecodebench; 16,384 for ceval and mmlu_redux; 15,000 for llm-system-prompts-benchmark; 49,152 for Claw-Eval.
Metrics — the columns mirror the table:
- Accuracy (Base / Ours) — fraction correct (exact/regex match; soft compliance for llm-system-prompts-benchmark; judge task-score for Claw-Eval; judge CORRECT/INCORRECT for AA-LCR).
- Thinking tokens (Base / Ours) — mean length of the single-turn <think> trace (think-per-task for Claw-Eval).
- Reduction — the average per-question thinking-token saving: base and Ours are paired on the same question (each side seed-averaged), each question's (base − cap)/base is taken, then averaged over shared questions (a larger ↓ = a bigger saving).
- Macro average (bottom row) — equal-weight mean across benchmarks.
We separately track two trace-quality failure modes, reported only in aggregate: looping — the model gets stuck repeating the same reasoning chain (sometimes a single sentence), never finishing its thinking; detected from the fraction of repetitive n-grams — and truncation — the <think> trace never closes because the model hits the generation-token cap while still reasoning, so no answer is produced. Across all out-of-domain responses, truncation drops from 2.9% to 0.4% while looping stays negligible (~0.2%).

In-domain evals

Holdout test splits of datasets whose train splits are part of the finetuning mix — quality retention on in-distribution tasks (in contrast to the out-of-domain benchmarks above).

Settings

Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
Decoding: sampling temperature=1.0, top_p=0.95, top_k=20, min_p=0.0 (bottlecapai/ThinkingCap-Qwen3.6-27B uses the base's sampling).
Max generation tokens: 15,000 for GSM8K; 8,192 for the MCQ sets.
Data: GSM8K is the full 1,319-row test split; the MCQ sets are capped at 1,000 rows (OpenBookQA = 500 and QASC = 926 are smaller, so full).
Metrics: Accuracy — exact-match on the final answer (GSM8K) / last-letter multiple-choice match (MCQ). Thinking tokens, Reduction and Macro average are as defined for the token-efficiency table above, as are the looping and truncation failure modes: across all in-domain responses, truncation drops from 1.6% to 0.03% while looping is negligible for both (≤0.01%).

Guardrails preservation

Settings

Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
Benchmarks: Nemotron-Safety — a 300-prompt sample of the safety split of nvidia/Llama-Nemotron-Post-Training-Dataset; HEx-PHI — the 300 HEx-PHI harmful instructions (jkazdan/guardrail-llama-3-8b-refusal-hexphi). Both are safety-relevant / jailbreak prompts.

Both datasets only contain a train subset and given their near-perfect scores, it is hard to rule out possible contimination in Qwen's original training data. These results are intended purely as a demonstration of preserved guardrail capabilities, and are in line with our internal usage testing.
Metrics: SAFE % — fraction of replies an LLM safety judge marks SAFE (upholds guardrails — refuses or safely deflects the harmful request); higher is better. Thinking tokens and Reduction are as defined for the token-efficiency table above.

Usage

HuggingFace Transformers

python
from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B", dtype="bfloat16")
proc = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B")

Check https://huggingface.co/Qwen/Qwen3.6-27B for recommended usage, sampling params etc.

FP8 (vLLM / SGLang)

bash
vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

GGUF (llama.cpp)

Quantized GGUF builds of this model live in the sibling repo bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF, for local inference with llama.cpp and compatible runtimes (Ollama, LM Studio, …).

bash
llama-cli -hf bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M -p "Hi"

Where to find us

Citation

If you use this model, please cite:

bibtex
@misc{ThinkingCap-Qwen3.6-27B,
  title     = {bottlecapai/ThinkingCap-Qwen3.6-27B},
  author    = {Lasocki, Karol and Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Mikolov, Tomas},
  year      = {2026},
}

Acknowledgements

We acknowledge EuroHPC Joint Undertaking for awarding the project ID EHPC-AIF-2025SC03-029 access to Leonardo at CINECA, Italy.