GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
nbeerbower
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
nbeerbower
Model Tree
nbeerbower/Qwen3.6-27B-TIES
Input Modalities
Output Modalities
Supported Functionality
| level | TIES | + Schierling | |
|---|---|---|---|
| L1 | syntax | 1/9 | 8/9 |
| L2 | stdlib | 0/5 | 1/5 |
| L3 | algorithms | 0/7 | 5/7 |
| L4 | systems | 0/7 | 5/7 |
| L5 | translation | 0/5 | 3/5 |
| L6 | debugging | 3/5 | 4/5 |
| overall | weighted | 10.1% | 65.0% |
| tasks passed | 4/38 | 26/38 |
22 tasks fixed, 0 broken. Nothing the base could already do was lost.
The sharpest single number: generated code that fails to execute drops from 30 tasks to 8. The base does not write runnable Hemlock at all — its 10.1% comes almost entirely from L6, where the task supplies existing code to repair rather than asking for a program from scratch.
L2 (stdlib) is the weak spot at 1/5. Standard-library work needs exact function names and
signatures — memorised API knowledge rather than transferable syntax. The stdlib portion of the
training data (the formulary rows folded into codex3) is the smallest slice, and the per-task
cap trimmed it further. More stdlib coverage is the obvious next improvement.
An earlier attempt taught Hemlock with 400 ORPO preference pairs and did not work. Inspecting that dataset showed why: only 186 pairs differed in code — the other 214 differed in which tool was called, so the objective could be satisfied by fixing tool selection without emitting a single correct line of Hemlock. Preference optimisation sharpens choices among behaviours a model already has; it does not install a grammar it has never seen.
Same language, same base, two instruments: 400 preference pairs moved nothing, 5,562 supervised examples moved 55 points.
| base | nbeerbower/Qwen3.6-27B-TIES |
| method | SFT, LoRA r=32 α=64 dropout 0.05 |
| data | 5,562 rows |
| schedule | 2 epochs, 682 steps, lr 2e-4 cosine, warmup 0.05 |
| sequence | max_length 1536, max_prompt_length 768 |
| batch | 2 × grad-accum 8 = effective 16 |
| source | rows | contributes |
|---|---|---|
hemlang/Hemlock-SFT | 2,673 | docs, semantics, compiler/parity tests |
hemlang/hemlock-codex3-SFT | 923 | execution-verified generation/translation/stdlib |
hemlang/hemlock-transmutation | 1,966 | verified C/Python → Hemlock, templated |
Two filters the dataset cards do not apply:
bench_overlap rows dropped. Their task names collide with hembench scenarios. The
codex3 card suggests keeping them for SFT — fine if you never benchmark on hembench, and
invalidating if you do. Dropping them is what makes the 65.0% a held-out number.codex-SFT, codex2-SFT and formulary-SFT are not used separately — codex3 supersedes and
merges all three.
Training loss fell 0.863 → 0.231 across epoch 1, then stepped to 0.068 at the epoch boundary and stayed flat for the remaining 230+ steps. That is the shape of memorisation rather than learning, and it suggested the second epoch was wasted.
The benchmark says otherwise — 26/38 on held-out tasks is not what a memorised model produces.
Still, 1 epoch is worth trying: it would halve a 14-hour run. No epoch-1 checkpoint was kept
here to compare against, since Merlina's epoch-end save overwrites; set save_steps explicitly
if you want intermediates.