Results
Measured on hembench with every generated program executed by the real interpreter (Q8_0,
greedy). The comparison point is the sibling ...-TIES-Hemlock-SFT model (the same base
with a lighter Hemlock SFT); "after" is the plain TIES base with this adapter merged:
Table with columns: level, lighter SFT, this adapter| level | | lighter SFT | this adapter |
|---|
| L1 | syntax | 6/9 | 7/9 |
| L2 | stdlib | 1/5 | 1/5 |
| L3 | algorithms | 2/7 | 7/7 |
| L4 | systems | 2/7 | 4/7 |
| L5 | translation | 0/5 | 2/5 |
| L6 | debugging | 2/5 | 2/5 |
| weighted overall | | 28.5% | 57.1% |
| programs that fail to run | | 21 | 8 |
No measurable general-capability cost: ARC 61.5→62.5, wiki perplexity +1.9%.
As with the 27B, stdlib is the weak axis — exact function names and signatures are
memorised API knowledge; check calls against the docs.
Training
SFT via Merlina: LoRA r32/α64, dropout 0.05,
lr 2e-4 cosine, 2 epochs, bf16, max_length 2048, on a single RTX A6000 (718 steps,
final loss 0.090).
Merging caveat
The base is a Qwen3.5 VLM whose checkpoint carries 15 mtp.* tensors (the multi-token-
prediction head). peft merge_and_unload() silently drops them, and current llama.cpp then
refuses to load the merged model. Graft them back from the base's model-vlm.safetensors
after merging — they are untouched by a language-model LoRA, so the copy is exact.
Note on the base
Huihui-Qwen3.5-9B-abliterated-TIES also has a sibling with a lighter Hemlock SFT already
baked in (...-TIES-Hemlock-SFT, trained on the 2,673-row base set only — 28.5% hembench).
This adapter was trained on the plain TIES base and supersedes that rung.