hemlang

Inkling-Small-Hemlock-SFT-LoRA

README

License: apache-2.0

Datasets

Three hemlang corpora, merged to 4,059 conversations:

Table with columns: Dataset, Rows used, Content
Dataset	Rows used	Content
hemlang/Hemlock-SFT	891	Documentation Q&A about the language
hemlang/hemlock-codex3-SFT	1,202 of 1,363	Verified code generation and translation
hemlang/hemlock-transmutation	1,966	C (1,102) and Python (864) programs translated to Hemlock, each verified by matching expected stdout

Why only three of the nine hemlang datasets

The rest are redundant or not SFT data:

hemlock-codex-SFT (552) is a strict subset of hemlock-codex2-SFT — every row is contained in it.
hemlock-codex3-SFT is a verified rebuild of codex2 + hemlock-formulary-SFT: its source column names both, instruction overlap with codex2 is 1114/1114, and it adds expected_stdout, sandbox_ok, and bench_overlap. Outputs are also fenced as ```hemlock.
Hemlock2-DPO, Hembot-DPO, Poison-DPO are preference data, not SFT.

So docs + codex3 + transmutation covers the corpus with no duplicated supervision.

Benchmark decontamination

150 rows flagged bench_overlap: True were excluded, along with 11 with sandbox_ok: False. Those 150 overlap hembench, so training on them would make any hembench score partly memorisation with no way to tell. This adapter has not seen them.

Transmutation rows were rendered as translation prompts with three rotating phrasings ("Translate this C program to Hemlock", etc.), assistant responses fenced as ```hemlock. All examples carry the system prompt You are a Hemlock programming expert.

Corpus length: median ~217 tokens, p90 ~690, max ~2,517 — so max_length 4096 truncates nothing.

Training

Table with columns: Parameter, Value
Parameter	Value
Base	`thinkingmachines/Inkling-Small`
Method	LoRA SFT (Tinker Cookbook `supervised.train`)
LoRA rank / alpha / dropout	32 / 32 / 0
Learning rate	1e-4, linear decay
Epochs	2 (244 optimizer steps)
Batch size	32 conversations
Max length	4096
Renderer

Results

Table with columns: start, end
	start	end
train NLL	0.9025	0.1458
eval NLL	0.9995	0.1774 (min 0.1772)

Eval NLL fell monotonically for 230 steps, bottomed at 0.1772, and ticked up slightly on the final eval — so two epochs landed close to the natural stopping point and a third would likely have begun overfitting. The train/eval gap is small (0.146 vs 0.177), which argues the model learned the language rather than memorising 3,904 examples.

Serving

sabrewing (raw Tinker format — what this repo ships)

sabrewing applies Tinker adapters directly and wants the raw checkpoint, not the PEFT conversion (which materialises Tinker's shared factors per-expert at 3x the size for no gain):

sh
hf download hemlang/Inkling-Small-Hemlock-SFT-LoRA --local-dir ~/Models/hemlock_lora
hf download sabrewing-engine/Inkling-Small-colibri-int4 --local-dir ~/Models/inkling_small_i4

SNAP=~/Models/inkling_small_i4 ./c/inkling -l ~/Models/hemlock_lora \
  -p "Write a Hemlock program that reverses a string." -n 256

Residents (attention, dense MLP, shared experts, lm_head) are merged at load; routed experts keep A/B factors resident with a low-rank correction wrapping the int4 matmul, because a rank-32 delta is smaller than the int4 quantization step and baking it in would destroy the adapter signal.

Tinker

Sample directly from the training checkpoint with a SamplingClient.

Evaluation

Not yet benchmarked. Eval NLL measures next-token prediction on held-out Hemlock text — it does not establish that generated programs compile or run. hembench is the harness that would: 38 scenarios across syntax, stdlib, algorithms, systems/concurrency, cross-language translation, and debugging, executing each program in a Docker-isolated verifier against expected output. Because the bench_overlap rows were excluded, that would be a clean read.

Related: nbeerbower/Schierling-Qwen3.6-27B-LoRA teaches Hemlock to a 27B dense base from the same three corpora.

License

Apache 2.0, matching the base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

hemlang

Model Tree

Base

thinkingmachines/Inkling-Small

Adapter

this model

Input Modalities

TextAudioImageVideo

Output Modalities

Text

Supported Functionality