ISB369

shellminator-qwen05b-dpo-selfplay

README

License: apache-2.0

What this model is

A full fine-tune of Qwen2.5-Coder-0.5B-Instruct for translating natural language into one bash command, polished with DPO against its own execution-verified failures. Ships as safetensors and a Q4_K_M GGUF (served by llama.cpp or ollama).

The user-facing launcher that turns this into sm "<question>" lives here: github.com/ISB333/shellminator.

Quick start

Zero-setup (recommended):

bash
curl -fsSL https://raw.githubusercontent.com/ISB333/shellminator/main/install.sh | bash
sm "show me the 5 largest files in /var"

Plain ollama:

bash
huggingface-cli download ISB369/shellminator-qwen05b-dpo-selfplay \
  shellminator-qwen05b-dpo-selfplay-Q4_K_M.gguf --local-dir .
# note: FROM needs an ABSOLUTE gguf path (ollama rejects relative ones)
printf 'FROM %s\nPARAMETER temperature 0\nPARAMETER num_predict 200\nPARAMETER stop "<|im_end|>"\n' \
  "$(pwd)/shellminator-qwen05b-dpo-selfplay-Q4_K_M.gguf" > Modelfile
ollama create shellminator -f Modelfile
ollama run shellminator "count hidden files in the current directory"

Transformers:

python
from transformers import AutoModelForCausalLM, AutoTokenizer

SYSTEM = ("You are a helpful assistant that translates natural language to bash commands.\n"
          "Context: cwd=/home/user, system=Linux x86_64, shell=bash.\n"
          "Reply with a single bash command only. No explanation, no markdown fences.")
tok = AutoTokenizer.from_pretrained("ISB369/shellminator-qwen05b-dpo-selfplay")
model = AutoModelForCausalLM.from_pretrained("ISB369/shellminator-qwen05b-dpo-selfplay").cuda()

prompt = "show kernel name, release and version"
text = tok.apply_chat_template(
    [{"role": "system", "content": SYSTEM},
     {"role": "user", "content": f"Generate single Bash command: {prompt}"}],
    tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=200, do_sample=False,
                     eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Benchmark

Table with columns: Model, IBM nl2bash-eabench bash_1 · 50 prompts · single-pass greedy · exec-verified
Model	IBM nl2bash-eabench bash_1 · 50 prompts · single-pass greedy · exec-verified
Qwen2.5-Coder-0.5B-Instruct (stock)	44%
gemma-3-270M fine-tune (first-gen shellminator)	48%
+ SFT on 105K exec-filtered rows (`shellminator-qwen05b-sft105k`)	72%
+ DPO self-play — 804 verified pairs (this model)	78%

benchmark

Protocol: each prompt's command runs in a per-test docker sandbox and is graded by the IBM eabench verifier (exit code + workspace state). One greedy pass, no retries, no self-correction, temperature 0.

What the +6 points bought (fail classes fixed end-to-end): find-count-without-self (find . -mindepth 1 -type d | wc -l), df -i overlay inode read-outs, uname flag selection. All six fail classes the SFT missed are represented in the 804 pairs (find_count 263, uname_flags 144, sed_insert_top 144, mv_glob_dir 77, …).

How it was made (exec-verified self-play)

Three data levers were measured at +0 against the 72% SFT plateau: +12K rows, teacher distillation DPO (synthetic rejections), and 949 targeted rows. The lever that worked:

Build a 9,219-prompt pool weighted toward the residual fail classes, keeping 105K provenance (training/build_dpo_pool.py).
Harvest the model's greedy outputs on Modal A100 (~$2): the gold command AND the generated candidate run in fresh per-row sandboxes; exit code, stdout, stderr and the post-state file tree must match, gold first. Divergent rows where gold itself broke or was unreproducible were dropped — including a two-run self-consistency probe that kills free/df-style drift noise (without it, ~8% of harvest smoke pairs punished equally-correct answers).
One DPO epoch (TRL, lr 5e-7, beta 0.1 → 26 s of A100): chosen = gold, rejected = the model's own wrong command. Eval accuracies 0.90 / margins 1.29.

Full recipes and scripts: github.com/ISB333/shellminator → training/. A chained round 2 (477 pairs from the upgraded champion) scored 74% — an ablation kept private; preference tuning past the first big-class dose overfits recent pairs.

Limitations

Single command by design — no multi-line scripts, no heredocs; the launcher's refine loop is the intended workflow for chaining.
Residual fails: sed -i '1i …'-style top inserts inside for-loops (the model can overwrite the file), comm/diff count variants, mv dir1/* dir2/ arg order. These survived +12K rows, synthetic DPO, targeted SFT, and two rounds of self-play.
Trained on Linux-x86_64-conventional fixtures; commands referencing exotic mounts (overlay, xvda) inherit sandbox assumptions.
The benchmark's protocol (single-pass greedy, exec-verified) is documented in the repo; numbers are not comparable to token-match nl2bash scores.

Training config

Table with columns: Stage, Value
Stage	Value
Base	Qwen2.5-Coder-0.5B-Instruct
SFT	105K conversational rows, 3 epochs, lr 2e-5, assistant-only loss, fp32 + bf16 autocast
Pool	9,219 fail-class prompts (provenance: part + line)
Harvest	batch 64, bf16, greedy, 200 tokens, exec-verify both sides, 24 workers
DPO	804 pairs, 1 epoch, lr 5e-7, beta 0.1, 48 steps, A100
GGUF	convert_hf_to_gguf f16 → Q4_K_M (398 MB)

Citation

bibtex
@misc{shellminator2026,
  title  = {shellminator: exec-verified self-play for a 0.5B bash coder},
  author = {ISB333},
  year   = {2026},
  url    = {https://github.com/ISB333/shellminator}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ISB369

Model Tree

Base

ISB369/shellminator-qwen05b-sft105k

Fine-tuned

this model

Input Modalities

Text

Output Modalities