Results
Table with columns: axis, cases, TIES, Vorarbeiter, Wichtel| axis | cases | TIES | Vorarbeiter | Wichtel |
|---|
| delegate | 10 | 0/10 | 7/10 | 10/10 |
| no_tool (restraint) | 10 | 9/10 | 9/10 | 10/10 |
| trivial (one-line edits) | 5 | 4/5 | 5/5 | 5/5 |
| inspect | 5 | 5/5 | 5/5 | 5/5 |
| exec | 4 | 4/4 | 4/4 | 4/4 |
| git | 4 | 4/4 | 4/4 | 4/4 |
| memory | 3 | 3/3 | 3/3 | 3/3 |
| web | 3 | 3/3 | 3/3 | 3/3 |
| persona | 3 | 3/3 | 3/3 | 3/3 |
| total | 47 | 35/47 | 41/47 | 47/47 |
47/47 valid tool calls, zero hallucinated tool names. Greedy decoding, so these are
deterministic.
Hemlock — 67.6% on hembench
Every generated program is executed by the real interpreter and its stdout compared against
expected output. Not string matching.
Table with columns: level, TIES, Schierling/TIES, Wichtel| level | | TIES | Schierling/TIES | Wichtel |
|---|
| L1 | syntax | 1/9 | 8/9 | 7/9 |
| L2 | stdlib | 0/5 | 1/5 | 2/5 |
| L3 | algorithms | 0/7 | 5/7 | 4/7 |
| L4 |
Why this combination
No code agent understands Hemlock. Vorarbeiter's delegation policy hands engineering work to a
coding agent, which is correct for every language that agent knows and wrong for this one — if
the operator does not write Hemlock itself, nothing does. So this model has to do both: delegate
ordinary engineering, and write Hemlock directly.
The interesting result is that the two capabilities did not fight. The risk was concrete:
the Hemlock adapter is 5,562 rows of direct code generation with zero tool calls, exactly the
data that could teach an operator to write code instead of delegating. Instead delegation went
from 7/10 to 10/10.
An earlier attempt to get both from a single ORPO run failed in a specific, measurable way:
folding 400 Hemlock preference pairs into the delegation data dropped restraint from 9/10 to
4/10, because all 400 rows call a tool and nothing in the training set teaches answering
directly. Splitting the objectives — SFT for the language, preference training for the behaviour
— fixed it and taught the language far better besides (400 preference pairs moved hembench
approximately nothing; 5,562 supervised rows moved it 55 points).
Lineage
Qwen/Qwen3.6-27B
└─ huihui-ai/Huihui-Qwen3.6-27B-abliterated
└─ nbeerbower/Qwen3.6-27B-TIES 5-way TIES merge
└─ + Bubba-3ep (GreatFirewall ORPO — states facts it otherwise sanitises)
└─ + delegation LoRA (hands engineering work to a code agent) = Vorarbeiter
└─ + Schierling (Hemlock SFT, 5562 rows) = Hemlock-Qwen3.6-27B
└─ + egirl LoRA (delegation + Hemlock tool use, ORPO) ← Wichtel
Four adapters deep, all four capabilities intact.
MTP head
peft merge_and_unload() silently drops the 15 mtp.* tensors and current llama.cpp then
refuses to load the model. They are grafted back from the pre-merge base and verified
bit-identical. 1,199 tensors total (850 language, 333 vision, 15 MTP, 1 lm_head).
llama-server -m Wichtel-Q8_0.gguf -ngl 99 -c 8192 --jinja -fa on -np 1 \
--spec-type draft-mtp --spec-draft-n-max 4
MTP speculation is not bit-identical in practice — on this bench one decision flipped between
speculative and non-speculative decoding at temperature 0, apparently from numerical differences
between batched verification and sequential decoding flipping a near-tie.
Limitations
- 47/47 is a saturated benchmark. It cannot distinguish this model from a better one, and 47
cases is a small sample — treat it as "no remaining failures we can see", not as a ceiling.
The bench was widened from 10 cases to 47 during this work precisely because the 10-case
version could not resolve the differences being claimed with it.
- stdlib is the weak Hemlock axis (2/5). Standard-library work needs exact function names and
signatures — memorised API knowledge rather than transferable syntax. Check calls against docs.
- Single-turn tool use. It learns the first action, not what to do with results or how to
recover from a failed call.
- Delegation is bound to egirl's tool schema (
code_agent, glob_files, git_status…).
A different tool set needs different training data.
- The Chinese-language censorship gap persists from the Bubba layer: asking 六四事件是什么?
in Chinese remains the weakest item.
- Prose is untested here. A sibling experiment found that a prose adapter trained on this
family's own outputs degraded the model it was trained against — see
Obscura.