ManniX-ITA

Qwen3.6-27B-A3B-Coder

README

License: apache-2.0

Recipe

Competence map: the 256e teacher is profiled per-expert on a balanced corpus + targeted LiveCodeBench and MultiPL-E (Rust/Java/JS) PASS-response classes.
Drop map: wmax aggregation with the LCB + MPE classes up-weighted (1.5) → 72/256 experts dropped per layer, protecting the code-competent experts.
Top-10 routing (num_experts_per_tok = 10) baked into the config → the shipped default. Pass --override-kv qwen35moe.expert_used_count=int:8 to any llama.cpp tool to A/B back to native top-8.

Evaluation (Q6_K, llama.cpp, temp 0.6 / top-p 0.95 / top-k 20)

Table with columns: Benchmark, This model, Qwen3.6-35B-A3B (256e)
Benchmark	This model	Qwen3.6-35B-A3B (256e)
GPQA-Diamond	0.773	0.833
MATH-500	0.620	0.730
AIME	0.733	0.633
LiveCodeBench (v6, 77q)	0.688	0.714
IFEval	0.730	0.960
HumanEval	0.970	0.970
GSM8K	0.970	0.960
ARC-Challenge	0.944	0.935
MultiPL-E (top-8, see note)	0.870 ± 0.02	0.863
Average	0.811	0.844

Highlights: best code profile of any prune — MultiPL-E 0.870, at parity with the 256e teacher (0.863) on the same measurement, LiveCodeBench 0.688, HumanEval 0.970. Average 0.811 sits within 0.035 of the full teacher despite dropping 72 of 256 experts per layer.

MultiPL-E re-measured, 2026-08-20. This row previously read 0.840 / 0.827. Our chat-reply → function-body extractor had a fallback that cut at the first { in the reply, which ate the opening line of a body-only answer and turned working code into a compile error. Re-scoring the same stored generations with the fixed extractor lifts both columns, concentrated in Java. The defect was in our harness, not in either model.

Two caveats, stated rather than hidden. (1) The cut was measured twice under an identical configuration and scored 0.850 and 0.890 — a 4pp spread on 300 problems. The value above is their mean with that spread as the band; a single MultiPL-E draw at this sample size should not be read to three digits. (2) These runs served the GGUF at its file default of expert_used_count = 8, whereas the published GGUF ships top-10. So this row measures the top-8 routing of the same weights, not the shipped default. A matched top-10 re-measurement is in progress and this row will be replaced when it lands.

Verbosity / rumination (length breakdown per eval)

Aggressive expert pruning makes the model verbose on open-ended reasoning — it over-thinks before answering. This is largely inherited from the base (the 256e teacher does the same on GPQA/AIME) and is bounded by the generation cap; it does not affect the code benches, which have a natural termination anchor.

Response length in characters (content + reasoning), this model vs the 256e teacher; runaway = responses > 20k chars (of 100, or 30/198 for GPQA, 30 for AIME):

Table with columns: Benchmark, p50, p90, max, runaway, 256e runaway
Benchmark	p50	p90	max	runaway	256e runaway
GPQA	13.8k	58.8k	129k	58	58 (same)
AIME	52.9k	85.5k	96k	25	29
IFEval	12.1k	56.2k

Reading it: GPQA/AIME verbosity is essentially the base model (58 vs 58, 25 vs 29). Only IFEval shows prune-added rumination (30 vs 11) — the trade for the code-targeted drop map. Code and math-with-boxing tasks terminate cleanly. If you want tighter output, a repetition/length penalty at serve time (or top-8 via the override above) reduces the tail.

Reasoning budget and thinking stop phrase (llama.cpp)

Qwen 3.6 reasons at length by design, and on a hard prompt it can consume the whole context window before it answers. llama.cpp can bound the thinking block with a sampler, and — the part that actually matters — tell the model why the block is being closed.

Needs llama.cpp b8508 or newer for the flags, b10091 or newer for the per-request overrides.

Serve with a bounded thinking block

bash
llama-server -m Qwen3.6-27B-A3B-Coder-Q4_K_M.gguf -c 32768 -ngl 99 \
    --jinja \
    --reasoning-budget 8192 \
    --reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n' \
    --temp 0.6 --top-k 20 --top-p 0.95

Table with columns: flag, meaning
flag	meaning
`--reasoning-budget N`	`-1` unrestricted (default), `0` close the block immediately, `N > 0` cap it at N tokens
`--reasoning-budget-message`	text written into the block just before the closing tag is forced
`--jinja`	required — the delimiters come from the chat template (`<think>` … `</think>`). Without it llama.cpp has no tags to count and the budget silently does nothing

Both flags also read from the environment: LLAMA_ARG_THINK_BUDGET and LLAMA_ARG_THINK_BUDGET_MESSAGE.

--reasoning-format is not part of this. It only decides how the thinking is handed back — message.reasoning_content versus left inline in message.content — and never whether the budget is enforced: the delimiters the sampler counts are set by the chat template regardless, so the cap binds under auto, deepseek and none alike. The default auto already extracts reasoning and is behaviourally identical to deepseek (they differ only in name; the sole branch in the parser is != none). Leave it at the default so the model's own tool-call and channel handling stays in play, and pin deepseek only when a harness needs the thinking kept out of content.

--reasoning-budget on its own forces the closing tag the moment the budget runs out, wherever the model happens to be. When that lands mid-thought the model frequently does not register that it was interrupted: it carries on reasoning, now inside the visible answer. The stop phrase is what prevents that — it gives the model a reason to be finishing.

Two wordings that work

bash
# "qwen" — the string Qwen's own service uses, from their docs
--reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n'

# "voice" — shorter, in the model's own reasoning voice
--reasoning-budget-message $'\n\nOK, I have enough to answer now.\n'

Wording is model-specific: Qwen note that the ability to act on such a message "is not explicitly trained but emerges naturally", so it is worth trying both on your own workload. Leading and trailing newlines matter — they keep the phrase off whatever half-finished line the cut landed on.

What it measures out to

Measured on the Qwen3.6-35B-A3B base this model is pruned from. Three hard questions, temperature 0.6, fixed seed, answer characters with wall time in brackets. Every run answered all three correctly, and thinking length is unchanged by the message in every row:

Table with columns: budget, no message, qwen, voice
budget	no message	`qwen`	`voice`
2048	1907 (69 s)	1838 (42 s)	1615 (41 s)
4096	18015 (170 s)	2642 (78 s)	1441 (104 s)
8192	3642 (158 s)	1848 (129 s)	2023 (175 s)

The 4096 row is the failure this exists for: the cap lands mid-thought and the reasoning simply continues in the answer, ten times longer and 2.2x the wall time, for the same three correct answers. Both phrases remove it.

Per request, instead of per server

The server accepts both as request fields, overriding the command line:

json
{
  "messages": [ ... ],
  "thinking_budget_tokens": 8192,
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On the raw /completion endpoint the delimiters are not inferred, so they have to be supplied with the budget:

json
{
  "prompt": "...",
  "reasoning_budget_tokens": 8192,
  "reasoning_budget_start_tag": "<think>",
  "reasoning_budget_end_tag": "</think>",
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On b10091 the message field must be present on /completion requests even when empty: llama.cpp builds the sequence it forces from message + end_tag inside that field's handler, so omitting it leaves the budget with nothing to force — the sampler logs as though the cap fired while the thinking block stays open.

Rules of thumb

Keep -c several times larger than the budget. A budget equal to the context lets the thinking phase fill the window on its own.
A quarter of the context is a sensible starting point: 8192 at -c 32768.
Qwen recommend keeping a thinking budget above 1024 tokens; below that the cap tends to land before the model has committed to an approach.
The budget is per thinking block, not per response — the sampler re-arms when it sees a new opening tag, so a multi-turn agent gets a fresh window each time.

Formats

GGUF (this repo family): full imatrix quant sweep (Q8_0 → IQ2, plus ContribDynamic CD-* per-layer quants) in Qwen3.6-27B-A3B-Coder-MTP-GGUF. Includes the native MTP head (speculative decoding) and a -vision mmproj for multimodal use. imatrix.dat archived in-repo.
Ollama: mannix/qwen3.6-27b-a3b-coder (text) and …-vision tags (with mmproj).

Notes

Top-10 is baked as the default; the model was selected and evaluated at top-10.
Same tokenizer, chat template, MTP head and vision tower as the base.
Research checkpoint. Verbosity on open-ended prompts is a known, base-inherited trait.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

ManniX-ITA

Model Tree

Base

Qwen/Qwen3.6-35B-A3B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality