Summary
mini-agent-v2 is a QLoRA fine-tune of Qwen2.5-Coder-1.5B-Instruct, trained jointly on
function calling, Bash and Python on a single 8 GB consumer GPU. It is small enough to run
on a laptop. It ships as merged bf16 weights and as GGUF builds from Q3_K_M to bf16.
It speaks the standard tool-call protocol: Qwen's own chat template with
<tool_call>{"name": ..., "arguments": {...}}</tool_call>. That is the format llama.cpp
(--jinja), vLLM, Ollama and OpenAI-style clients already parse. You don't need a custom
system prompt or parser: pass your tools in the normal tools field. On the Berkeley Function
Calling Leaderboard (BFCL) questions it answers 86.2% correctly, up from 52.7% for the
base model.
At a glance
Table | |
|---|
| ✅ Use it for | native tool calls through any OpenAI-compatible server · Bash one-liners · small Python functions · local/offline agents |
| ⚠️ Don't use it for | long autonomous multi-step agent loops · shell commands you won't read · anything safety-critical |
| 📈 Best result | 86.2% on BFCL non-live in FC mode (base: 52.7%), level with Qwen3-1.7B and xLAM-2-3B |
| 📉 Biggest weakness | real-user BFCL live questions: 65.7%, so about 1 in 3 is still wrong |
| 🧰 Works with | llama-server --jinja, vLLM, transformers; tool calls checked on every GGUF |
| 💾 Smallest build | 824 MB (Q3_K_M GGUF) |
Specifications
Table | |
|---|
| Base model | Qwen/Qwen2.5-Coder-1.5B-Instruct (1.54 B params, 28 layers, tied embeddings) |
| Method | QLoRA over unsloth/qwen2.5-coder-1.5b-instruct-bnb-4bit: LoRA rank 16, alpha 32, dropout 0, bias none |
| Trainable params | 18,467,840: 392 LoRA A/B tensors (28 layers × 7 projections) + 2 re-initialised tool-tag embedding rows |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, , |
Results
These are the three tasks the model was trained for, compared with the base model under an
identical harness. Every answer is executed, not string-matched: Bash runs in Docker,
Python runs against unit tests, and tool calls go through BFCL's AST checker.
Table with columns: Benchmark, n, Base Qwen2.5-Coder-1.5B-Instruct, mini-agent-v2, Δ, Paired test| Benchmark | n | Base Qwen2.5-Coder-1.5B-Instruct | mini-agent-v2 | Δ | Paired test |
|---|
| Tool calls: BFCL non-live, FC mode | 1000 | 52.7% | 86.2% | +33.5 pp | McNemar p < 0.001 |
| Tool calls: BFCL live, FC mode | 1335 | 56.9% | 65.7% | +8.8 pp | McNemar |
- Tool calling is the big gain. The base model's tool-tag embeddings are untrained, so it
barely emits its own
<tool_call> format, and when asked for tools it mostly answers in
Markdown JSON. Fixing that, plus the tool data, takes it from 52.7% to 86.2%.
- Bash and Python improve without being traded away. NL2SH-ALFA gains 26 commands and
Python gains 38 problems, both significant. MultiPL-E sh (writing whole shell programs)
is unchanged.
- InterCode needs a caveat. 127 of its 200 tasks overlap NL2SH-ALFA's test split, and the
whole gain sits there. On the 73 independent tasks it is flat.
Every model was run through the same harness in bf16, using its own chat template and native
tool-call format.
Table with columns: Model, BFCL non-live, BFCL live, NL2SH-ALFA, InterCode, MultiPL-E sh, HE+/MBPP+| Model | BFCL non-live | BFCL live | NL2SH-ALFA | InterCode | MultiPL-E sh | HE+/MBPP+ |
|---|
| mini-agent-v2 (1.5B) | 86.2% | 65.7% | 50.3% | 36.0% | 32.6% | 66.2% |
| Qwen2.5-Coder-1.5B-Instruct (base) | 52.7%* | 56.9%* | 41.7%* | 30.5% |
Notes: * = the gap to mini-agent-v2 is statistically significant (both models answered the same
questions; paired exact McNemar test, p < 0.05), so it is unlikely to be luck. No * means the two are
effectively tied. ** = that model was not tested on this benchmark (not a score of zero).
Files
The weights are split across two repos: F-A-I-L/mini-agent-v2 holds the bf16
safetensors for transformers / vLLM, and F-A-I-L/mini-agent-v2-GGUF holds the GGUF
builds for llama.cpp and GGUF apps.
Table with columns: File, Repo, Size, Bits per weight| File | Repo | Size | Bits per weight |
|---|
model.safetensors + config.json + tokenizer files | mini-agent-v2 | 3.09 GB | 16 (bf16) |
mini-agent-v2-Q3_K_M.gguf | mini-agent-v2-GGUF | 824 MB | 4.24 |
mini-agent-v2-Q4_K_M.gguf |
GGUF metadata: architecture qwen2, 1,543,714,304 params, context 32768, EOS <|im_end|>,
with the stock Qwen2.5 chat template (tool support intact). All quants were made from the
bf16 GGUF with llama-quantize. Plan on the file size plus about 1 GB of memory.
Scores above are for the bf16 weights. Fewer bits per weight means a smaller file and lower
quality. Every GGUF was checked to produce a correct tool call, but the quants were not
re-benchmarked, so expect somewhat lower scores than these, more so the smaller the file.
How to prompt it
Tool calling: pass tools through the chat template's tools= field (or the OpenAI
tools field on a server). No system prompt is needed; the template inserts the tool list.
It replies with one or more <tool_call>{"name": ..., "arguments": {...}}</tool_call>
blocks, or with plain text when no tool fits.
Bash
You are a Bash assistant. Translate the request into a single Bash command.
For a full script: You are a Bash assistant. Write a complete Bash script for the request.
Python
You are a Python coding assistant. Write correct code. Respond with a single code block.
The benchmark numbers on this card were measured with greedy decoding (temperature 0).
Usage
llama.cpp
# downloads the chosen quant from the GGUF repo on first run# (QUANT = Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0 or bf16)llama-server -hf F-A-I-L/mini-agent-v2-GGUF:QUANT --jinja -c 4096 -ngl 99# or, with a file you already downloadedllama-server -m mini-agent-v2-QUANT.gguf --jinja -c 4096 -ngl 99
--jinja is required for tool calling. Without it, llama.cpp ignores the template's tool
support.
curl localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{ "messages": [{"role": "user", "content": "What is the weather in Berlin in celsius?"}], "tools": [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": { "city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}]}'# -> tool_calls: get_weather({"city": "Berlin", "unit": "celsius"})
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer REPO = "F-A-I-L/mini-agent-v2"tok = AutoTokenizer.from_pretrained(REPO)model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto") tools = [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]msgs = [{"role": "user", "content": "Weather in Paris?"}]enc = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)out = model.generate(**enc, max_new_tokens=128, do_sample=False)print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=False))# <tool_call># {"name": "get_weather", "arguments": {"city": "Paris"}}# </tool_call><|im_end|>
Any GGUF app
LM Studio, Jan and KoboldCpp load the .gguf directly. To use tools, turn on the app's
tool/function-calling support.
🖥️ Built on one 8 GB laptop GPU, looking for a bigger one
Every run behind this model was trained and evaluated on a single RTX 4060 (8 GB). That is
why it's 1.5B, trained on a 4-bit base with 2k-token examples. The same weights already
score significantly higher on Bash and Python when run in full bf16 instead of 4-bit.
If your team has an NVIDIA DGX Spark or an AMD Ryzen AI Max+ ("Strix Halo") AI dev
kit to spare, it would go straight into full-precision training, longer context, 3B–7B
bases and verified-reward RL, all released openly.
Get in touch via Discussions.
No obligation; the model stays free either way. Thank you!
Evaluation
How this was measured
All results come from a run on 2026-10-04 (external models 2026-10-05), scored with each
benchmark's own scorer. Where a task can be executed, it was executed.
- Decoding: greedy, batch size 8, 4096-token eval context, bf16 weights, the same harness
for every model.
- Tool calling: BFCL non-live (simple,
multiple, parallel, parallel-multiple) and live, in FC mode. Tools go through each
model's
tools= chat template, the answer is decoded from its native tool-call output,
and it is checked with a reimplementation of BFCL's official AST checker. This is this
project's harness, not the public leaderboard's prompt mode, so the numbers are not
directly comparable with the leaderboard.
- Bash: NL2SH-ALFA test split,
executed through InterCode-ALFA in Docker and scored on filesystem/output equivalence.
InterCode-Bash runs in its own images.
MultiPL-E sh runs unit tests in a network-less container.
- Python: EvalPlus HumanEval+ and MBPP+ (base +
hardened "plus" tests), executed.
- Significance: paired exact McNemar test on per-question outcomes. Same problems, same
prompts, same decoding.
Table with columns: Split, mini-agent-v2, Base| Split | mini-agent-v2 | Base |
|---|
| non-live micro / macro | 862/1000 (86.2%) / 85.1% | 527/1000 (52.7%) / 43.7% |
| — simple / multiple | 362/400 · 181/200 | |
| — parallel / parallel-multiple | 162/200 · 157/200 | |
| live micro / macro | 877/1335 (65.7%) / 56.3% | 759/1335 (56.9%) / 29.0% |
| — live simple / live multiple | 160/258 · 697/1037 | |
| — live parallel / live parallel-multiple |
The remaining errors are mostly argument values, not tool choice:
- Optional parameters left out or added when the user didn't ask for them
- User wording not mapped onto a schema enum value
- Date arithmetic ("next Tuesday")
- Very long tool lists
Development history
mini-agent-v2 is run b15 of a series:
- b1–b9 trained BFCL's prompt format (
[f(a=1)]). It scored well on the benchmark, but
no inference server parses it.
- From b10, every run is native-FC only. b10 also brought the tool-tag fix: in the base
model, the
<tool_call> / </tool_call> embedding rows are byte-identical untrained
vectors with tied embeddings. LoRA doesn't touch embedding rows, so both were
re-initialised and trained as extra tokens.
- b11–b15: each run fixed one failure class found by row-level error analysis of the
previous run.
- b12 reverted a rule that taught dropping optional parameters and removed rows with
invented years.
- b14 added targeted slices.
- b15 removed the b14 slice that caused its non-live regression.
- b16 added longer context and a "fill discipline" filter. It came out null against b15,
so b15 is the release.
The chart shows each run's 4-bit evaluation, which is why b15 reads 85.7% / 65.8% there and
86.2% / 65.7% in bf16 above. Neighbouring runs are often within noise: b15 vs b14 was
p < 0.001 on non-live and p = 0.128 on live.
Training recipe
Table | |
|---|
| Epochs | 1 |
| LR / schedule | 1e-4, cosine |
| Batch | 4 × grad-accum 2 (effective 8) |
| Sequence | max 2048, packing=False |
| Regularization | none (NEFTune off) |
| Loss masking | train_on_responses_only, anchored on the user header so tool results are never trained as model output |
| Seed | 3407 |
| Framework |
Data: 24,745 rows after the length filter: tool 49%, Python 28%, Bash 23%.
- Tool (12,275, all native FC, rendered through the real chat template):
- xLAM calls, weighted toward parallel and multi-call shapes
- Toucan MCP calls from real servers
- 1,000 ToolACE rows
- 1,000 full glaive loops (call → tool result → answer)
- 500 "irrelevant" rows, where the tools are listed but the right answer is plain text
- Slices for array arguments, enum mapping, date phrasing and a generic shell tool
- No trained call contains a year the user didn't state.
- Python (7,000): LeetCode problems and stub completions, plus KodCode rows (CC-BY-NC-4.0, see License) whose answers
were re-executed against their unit tests.
- Bash (5,670): NL2SH-ALFA train,
bash-instruct-III one-liners and scripts, and
linux-command-dataset.
- Leak protection: training prompts that match BFCL, NL2SH-ALFA test, HumanEval or MBPP+
were removed.
Known limitations
- About 1 in 3 real-user tool questions is wrong. See the error classes above.
- Not a multi-step agent. Training is mostly single-call, with 1,000 two-hop loops. On a
Docker agent probe it completes 7/54 tasks, and 8/45 on held-out task families.
- Bash output is a single command or a short script, unverified. Sandbox it.
- The quants were not re-benchmarked. Expect lower scores than the bf16 numbers, more so the smaller the file.
- English only. No safety tuning beyond the base model.
Intended use and limits
Good for: local and offline tool-calling back-ends behind any OpenAI-compatible server,
shell one-liner translation, small Python functions, and edge deployment.
Not for: production automation without validation, long autonomous agent loops, or
anything safety-critical. Never execute generated shell commands unreviewed.
Citation
@misc{mini_agent_v2, title = {mini-agent-v2: a 1.5B native tool-calling, Bash and Python fine-tune of Qwen2.5-Coder-1.5B-Instruct}, author = {Frost2o24}, year = {2026}, url = {https://huggingface.co/F-A-I-L/mini-agent-v2}, note = {Merged bf16 weights and GGUF builds; evaluated on BFCL (FC mode), NL2SH-ALFA, InterCode-Bash, MultiPL-E sh and EvalPlus}}
License
The model weights are released under Apache-2.0, inherited from Qwen2.5-Coder-1.5B-Instruct.
⚠️ Training-data notice: about 5,100 of the 24,745 training rows (the KodCode Python
slices: 4,300 general + 800 odd-function-name rows, via F-A-I-L/kodcode-verified-python-235k)
come from KodCode-V1, which is licensed
CC-BY-NC-4.0 (non-commercial). All other training sources are Apache-2.0, MIT or CC-BY-4.0.
Whether a dataset's license carries over to a model trained on it is legally unsettled.
If you plan to use this model commercially, review that dataset's terms and decide for
yourself; this notice is not legal advice.
Training data licenses are those of the upstream sources listed in the metadata. xLAM
(CC-BY-4.0) is credited to Salesforce Research.