Goal
One 32 GB card (RTX 5090), 8-bit weights, 128k context. Protect: math, tool calls, reasoning,
common sense, spatial reasoning, coding. Allowed to lose: fact and history recall (who was
president when). Also wanted: fewer tokens spent on simple tasks.
The starting estimate was that the uncut model at Q8_0 would leave room for only ~30k context,
so ~4B parameters had to go. That estimate was later load-tested and was wrong (see the end).
Kept vs the original
Original = uncut Qwen3.8-27B, Q8_0. Kept = this model's score / the original's score.
Table with columns: Skill, Original, This model, Kept| Skill | Original | This model | Kept |
|---|
| math | 98 | 96 | 98% |
| tools | 91 | 92 | 101% |
| logic | 99 | 93 | 94% |
| common sense | 88 | 88 | 100% |
| spatial (text) | 89 | 86 | 97% |
| screen clicks (vision) | 91 | 89 | 98% |
| mean of the six | 92.7 | 90.7 | 97.8% |
| HumanEval | 100 | 97 | 97% |
| hard code (LiveCodeBench) | 48 | 32 | 67% |
| facts (allowed to drop) | 23 | 12 | 52% |
| same next token as the original (llama.cpp, held-out text) | | 93.3% | |
| parameters / Q8_0 file | 27.3B / 29.05 GB | 23.2B / 24.7 GB | 85% |
Full results
Same judge for all three: llama.cpp b11675, Q8_0 GGUF, reasoning effort low, sampling
temperature 1.0 / top-p 0.95 / top-k 20, 6,000-token cap per reply, 100 items per task (150 for
benign). Noise is about +-3 points near 90% and +-5 near 30-50%.
Table with columns: Task (source), Uncut 27B, Cut, Cut + heal (this)| Task (source) | Uncut 27B | Cut | Cut + heal (this) |
|---|
| math (GSM8K test) | 98 | 98 | 96 |
| tools (glaive function calling, judge items excluded from all training) | 91 | 92 | 92 |
| logic (BIG-Bench Hard) | 99 | 91 | 93 |
| common sense (CommonsenseQA, WinoGrande) | 88 | 86 | 88 |
| spatial, text (BBH navigate, StepGame) | 89 | 88 | 86 |
| screen clicks (ScreenSpot-v2 test, vision) | 91 | 90 | 89 |
| mean of the six above | 92.7 | 90.8 | 90.7 |
| code (HumanEval, code is run) | 100 | 100 | 97 |
| hard code (LiveCodeBench 50 medium + 50 hard, code is run) | 48 | 29 | 32 |
| facts (PopQA, allowed to drop) | 23 | 12 | 12 |
| refuses harmful (JailbreakBench) | 93 | 94 | 93 |
| refuses weapons (ALERT) | 41 | 35 | 36 |
| refuses sexual content (ALERT, CategoricalHarmfulQA) | 59 | 65 | 61 |
| refuses benign by mistake (JailbreakBench benign, XSTest safe; lower is better) | 11.3 | 12.7 | 12.7 |
| tokens per reply, mean | 710 | 768 | 759 |
| replies that hit the 6,000 cap (of 1,450) | 50 | 72 | 62 |
| hard code: replies that hit the cap (of 100) | 48 | 66 | 57 |
With a 16,000-token cap on hard code (only the capped replies redone) the uncut model went
48 -> 67 and the cut (before healing) 29 -> 45: the gap grows, so the cut lost skill, not only
budget. Not measured for the healed model.
How the judge scores: math and logic by the final answer, tools by exact function name and
arguments, screen clicks by whether the point lands in the target box, code by running each reply
against the tests in a separate short-lived process. Refusal by keywords at the start of the
reply; those columns are there to see drift, nothing was trained toward or against refusal.
What changed from the base model
- Every FFN is narrower: 17,408 -> 13,312 (all 64 blocks and the MTP block), 27.3B -> 23.2B
parameters. No other part of the architecture changed.
- A distillation adapter, trained against the uncut model, is merged into the weights.
- The vision tower is unchanged. The MTP block was not part of the healing.
- Pass mark, set before healing: hard code >= 40, six-skill mean >= 92, HumanEval 100. Result
32 / 90.7 / 97: not reached.
The method itself is not published.
Token use
The biggest saving found was a setting, not a weight change: reasoning effort low instead of
the template's default xhigh spent ~60% fewer tokens with equal or better accuracy on every
model tested. The waste is the tail (replies that run to the cap), not the typical reply.
Use
Same chat template and settings as Qwen3.8-27B. Recommended: reasoning effort low.
llama-server -m Qwen3.8-23B-Cut-Heal-Q8_0.gguf --jinja -ngl 99 -c 131072 \ --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on \ --chat-template-kwargs '{"reasoning_effort":"low"}'
Add --reasoning-budget N for a hard cap on thinking tokens. Transformers: load like
Qwen3.8-27B (Qwen3_5ForConditionalGeneration); intermediate_size in the text config is 13,312.
What the uncut model does on the same card (for comparison)
Measured on an RTX 5090 (32 GB), llama.cpp b11675, uncut Qwen3.8-27B Q8_0, q8_0 KV cache, no
vision projector, -ngl 99 --fit off: 128k context fits (31.95 of 32.6 GB), 50.8 tok/s, a
needle at 121k tokens found; 64k context with the MTP draft gives 84.8 tok/s. llama.cpp's default
--fit on moves layers to the CPU at 128k and drops to 19 tok/s. If the uncut model fits your
card, use it.
Limits
- One judge run per arm, one model, one recipe. No human evaluation.
- Fact recall was removed on purpose (PopQA 23 -> 12).
- The MTP (speculative draft) block is the cut one, not healed; its acceptance rate was not
measured.
- Languages other than English were not judged.
Licence
Apache-2.0, as the base model. Changes from the base: narrower FFN layers and a merged distillation
adapter.