Results
Ten operator requests in egirl's <tool_call> dialect, with its real system prompt:
Table with columns: metric, Qwen3.6-27B-TIES, + this LoRA| metric | Qwen3.6-27B-TIES | + this LoRA |
|---|
| tool + args correct | 7/10 | 8/10 |
| delegated coding work | 0/3 | 3/3 |
| valid tool calls | 10/10 | 10/10 |
| no hallucinated tools | 10/10 | 10/10 |
| terminated | 10/10 | 10/10 |
All three delegation cases flipped from exploring to handing off:
Table with columns: request, before, after| request | before | after |
|---|
| "split the auth module, make sure tests pass" | glob_files, glob_files | code_agent |
| "fix the Discord 2000-char bug" | glob_files, glob_files | code_agent |
| "add retry logic to every HTTP call, plus tests" | glob_files, |
Why this needed training
Delegation is not something prompting fixes. Every model benched scored 0/3, including with a
system prompt that says "Delegate coding work to the code agent by default — you're the
human-in-the-loop, not a code generator":
Table with columns: model, delegated| model | delegated |
|---|
Qwen/Qwen3.6-27B | 0/3 |
Qwen3.6-27B-TIES | 0/3 |
Huihui-ThinkingCap-Qwen3.6-27B-abliterated | 0/3 |
nightmedia/…-Architect-Polaris2-Fable-B-F451 | 0/3 |
| two other local merges | 0/3 |
gpt-5.6-sol | |
Swapping a generic bench prompt for egirl's real one moved overall tool accuracy 6/10 → 7/10 and
changed delegation not at all.
Regressions
Two cases got more eager to act, which is the expected failure direction for this axis:
Table with columns: case, before, after| case | before | after |
|---|
| "what's in /etc/hostname?" | read_file | execute_command |
| "in one sentence, TIES or SLERP?" | no tool (correct) | web_research |
The dataset has a 45-row band of single-line-edit examples meant to hold the "just do it yourself"
boundary, against 121 delegate rows. That ratio looks too delegate-heavy; a rebalanced mix is the
obvious next iteration.
Censorship eval also shifted on the same base — ccp_truth_neutral 4.00/4 → 2.40/4, total
29.00 → 26.60. Training on tool-use is not supposed to touch that axis, so treat it as evidence
that a narrow LoRA at r=32 still perturbs unrelated behaviour. Stack with
Bubba-3ep if you want the
censorship behaviour back, and re-measure.
Usage
llama.cpp, no merging required:
python convert_lora_to_gguf.py --base /path/to/Qwen3.6-27B --outtype f16 \
--outfile delegation-lora-f16.gguf /path/to/Qwen3.6-27B-delegation-LoRA
llama-server -m Qwen3.6-27B-TIES-Q8_0.gguf --lora delegation-lora-f16.gguf -ngl 99 --jinja
PEFT:
from peft import PeftModel
from transformers import AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained("nbeerbower/Qwen3.6-27B-TIES", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "nbeerbower/Qwen3.6-27B-delegation-LoRA")
Training
Table | |
|---|
| method | ORPO (β=0.1) |
| base | nbeerbower/Qwen3.6-27B-TIES |
| data | 558 pairs (186 tasks × ~3 workspace-path variants) |
| LoRA | r=32, α=64, dropout 0.0 |
| targets | q,k,v,o,gate,up,down_proj |
| lr | 8e-6 cosine, 10% warmup |
| batch | 2 × 4 accum (effective 8) |
| epochs / steps |
Trained with Merlina.
The training prompt is deliberately minimal — identity, workspace, tool definitions, nothing
else. No delegation policy and no "call the tool immediately" instruction, so the model learns
when to delegate as behaviour rather than learning to obey a sentence in context. egirl's real
prompt does state the policy, and the two compose.
Limitations
- Bound to egirl's tool schema. The
chosen targets name code_agent, glob_files,
git_status and friends. A different tool set needs different data.
- Single-turn. It teaches the first action only — not what to do with results, when to follow
up, or how to recover from a failed call.
- 186 underlying tasks, each seen under ~3 workspace paths. Path memorisation is addressed;
task diversity is not.
- Over-eagerness, and the censorship shift noted above.