Hemlock-Qwen3.5-9B API & Inference Endpoint

Results — hembench 57.1%

Every generated program is executed by the real interpreter and its stdout compared against expected output (Q8_0, greedy):

Table with columns: level, score
level		score
L1	syntax	7/9
L2	stdlib	1/5
L3	algorithms	7/7
L4	systems	4/7
L5	translation	2/5
L6	debugging	2/5
overall	weighted	57.1%

For scale: the 27B TIES scored 10.1% before its Hemlock training and 65.0% after. This 9B lands at 57.1% — most of the 27B's Hemlock capability at a third the parameters.

No measurable general-capability cost from the adapter: ARC 61.5→62.5 (n=299), wiki perplexity +1.9%.

Limitations

stdlib is the weak axis (1/5), same as the 27B family: standard-library work needs exact function names and signatures — memorised API knowledge rather than transferable syntax. Check calls against the docs.
Tool-use restraint measured slightly below the pre-SFT base (8/10 vs 10/10 on a 47-case bench) — the known cost of direct code-generation SFT on an operator model. Preference training repairs this; it is the next rung of this model's ladder.

Serving

bash
llama-server -m Hemlock-Qwen3.5-9B-Q8_0.gguf -ngl 99 -c 8192 --jinja -fa on

The MTP head supports --spec-type draft-mtp speculative decoding.

Results — hembench 57.1%

Every generated program is executed by the real interpreter and its stdout compared against expected output (Q8_0, greedy):

Table with columns: level, score
level		score
L1	syntax	7/9
L2	stdlib	1/5
L3	algorithms	7/7
L4	systems	4/7
L5	translation	2/5
L6	debugging	2/5
overall	weighted	57.1%

For scale: the 27B TIES scored 10.1% before its Hemlock training and 65.0% after. This 9B lands at 57.1% — most of the 27B's Hemlock capability at a third the parameters.

No measurable general-capability cost from the adapter: ARC 61.5→62.5 (n=299), wiki perplexity +1.9%.

Limitations

stdlib is the weak axis (1/5), same as the 27B family: standard-library work needs exact function names and signatures — memorised API knowledge rather than transferable syntax. Check calls against the docs.
Tool-use restraint measured slightly below the pre-SFT base (8/10 vs 10/10 on a 47-case bench) — the known cost of direct code-generation SFT on an operator model. Preference training repairs this; it is the next rung of this model's ladder.

Serving

bash
llama-server -m Hemlock-Qwen3.5-9B-Q8_0.gguf -ngl 99 -c 8192 --jinja -fa on

The MTP head supports --spec-type draft-mtp speculative decoding.

Hemlock-Qwen3.5-9B

README

Results — hembench 57.1%

Limitations

Serving

Explore FriendliAI today

README

Results — hembench 57.1%

Limitations

Serving