Evaluation (held-out probes, greedy)
45 real ambiguous trajectories (seeded held-out split), each demanding one of
a specific tool family. Same seed/limit for all models.
Table with columns: model, fired, name_match, args_ok, format_exact, clarify| model | fired | name_match | args_ok | format_exact | clarify |
|---|
| base Qwen3.8-9B-Distill | 0/45 | 0/45 | 0/45 | 0/45 | — |
| v2 (real traces) | 41/45 | 28/45 | 25/45 | 25/45 | 2/2 |
| v25 (trigger-heavy) | 26/45 | 18/45 | 16/45 | 26/45 | 0/2 |
| v26 (this model) | 43/45 | 25/45 | 23/45 | 27/45 | 2/2 |
v26 fires more (43/45 vs 41/45) and formats more exactly (27/45) than v2,
with clarify intact (2/2). Cost: −3 name_match / −2 args_ok — the model
sometimes picks the wrong tool under semantic ambiguity (e.g. search_files
instead of tool_describe on a docs probe). Selection precision is the
target of the next recipe iteration.
The base distill contributes zero tool-calling (0/45); the entire
capability comes from the fine-tune.
Intended use
Research artifact for experimenters working on tool-call behavior in 9B-class
models — not a product. The tool_call XML envelope and tool schemas match
the Hermes agent runtime this was trained on.
What this repo contains
- The BF16 merged weights (this repo)
- All scripts that produced the model and dataset (
scripts/)
Reproducibility
All scripts that produced this model are in scripts/:
Table with columns: file, purpose, number it produced| file | purpose | number it produced |
|---|
build_v26.py | dataset build (v2-minus-lambda + SWE-rebench + APIGen + When2Call → 1,543 rows) | row count, mix |
qwen38_train_v26.py | QLoRA continue-train via Unsloth (1 epoch, 193 steps, loss 0.0552) | training loss/steps |
merge_qwen38_v26_pure.py | shard-streamed BF16 merge (pure torch; MTP head preserved; vocab patched) | 12 shards / 18.4 GB |
smoke_v25_merged.py + eval_fc_heldout.py + |
The scripts carry the author's machine paths (F:/models/..., C:/AI/...)
and run on this rig's ROCm stack (Unsloth 2026.8.18, torch 2.11 ROCm,
RX 7700 XT); adjust paths for your environment.
Convert/quantize: llama-rocmfpx fork, convert_hf_to_gguf.py --outtype bf16
→ llama-quantize Q4_0_ROCMFP4_FAST (see GGUF repo).
Quantized version
Q4_0_ROCMFP4_FAST GGUF (ROCmFPX format, AMD RDNA3 kernels) is published in
the companion repo. ROCmFPX quants target AMD ROCm inference; for portable
use, convert from the BF16 merge here.
Training data note
The training data itself is not published. The rows derive from the
author's own agent sessions, and some rows contain private strings
(hostnames, session identifiers). The scripts that build the dataset are
published; the data is not. The weights do not contain those strings.
Acknowledgements
Base model and license inherited from Empero/Qwen3.8-9B (Apache-2.0).
Training data derived from the author's own Hermes agent sessions plus
publicly-licensed teacher sources (SWE-rebench, APIGen-MT-5k, When2Call).