Scores (vs the Hemlock base it started from)
Table with columns: axis, Hemlock-3.8, B0-27B| axis | Hemlock-3.8 | B0-27B |
|---|
| egirl 47-case operator bench | 47/47 | 42/47 |
| kirabench (end-to-end task completion via egirl + Codex) | 6/6 | 6/6, faster |
| censorship (strict, single-sample) | 27/29 | 28/29 |
| safety asymmetry (refuses actual harm) | 2/2 | 2/2 |
| prose distance vs contemporary fiction | 0.528 | 0.525 |
| stance rate (has opinions) | 16.7% | 8.3% |
| hembench | 79.9% | 73.4% |
| ARC / wiki-clean ppl | 65.9 / 10.01 | 66.2 / 9.93 |
| identity | "Qwen" | Schneewolf Labs |
The egirl-bench dip is five known single-tool-call reflexes (three delegation prompts
answered with git_status, two trivial edits answered with a file search). They do not
reach task completion: on kirabench — six sandboxed engineering tasks driven through the
real egirl harness with Codex as the code agent, judged only by deterministic checkers —
B0-27B goes 6/6 and fixed a failing-test task ~9× faster than its base.
Notes
- Preference training via Merlina
(ORPO r32/α64 lr 8e-6); the substrate adapter was hand-merged into the weights
(delta surgery — vision and MTP tensors byte-identical to the base) before the
capstone trained on the result. 1,199 tensors verified at every merge.
- MahouMix was rebuilt on-policy: ChatML parsed out, rejected regenerated from the
substrate-merged model itself.
- Vision tower and the 15
mtp.* tensors are untouched from the base; mmproj included
in the GGUF repo. --spec-type draft-mtp works.
- This is the internal egirl-testing base. The consumer model built on it ships as
Familiar Ember.
llama-server -m B0-27B-Q8_0.gguf -ngl 99 -c 16384 --jinja -fa on -np 1 \
--spec-type draft-mtp --spec-draft-n-max 4