Robokid1134

TinyToolRouter-0.6B

README

License: apache-2.0

Key features

Small footprint: 0.6B parameters, suitable for CPU-first inference.
Schema-aware routing: Selects from tool definitions supplied at inference time.
Structured output: Produces a single JSON object representing a proposed tool call.
Explicit abstention: Supports a dedicated no_tool response.
Paraphrase robustness: Evaluated on requests phrased differently from the training examples.
Unseen-schema evaluation: Tested on tools absent from the training set.
Multiple formats: Safetensors and GGUF builds are available in this repository.

Model details

Table with columns: Property, Value
Property	Value
Model	TinyToolRouter-0.6B
Base model	Qwen/Qwen3-0.6B
Parameters	0.6B
Fine-tuning	Supervised fine-tuning with LoRA (PEFT)
Deployment	Merged fine-tuned weights
Primary language	English
License	Apache-2.0
Intended hardware	CPU-only and low-VRAM systems
Safetensors size	Approximately 1.2 GB
GGUF Q4_K_M size	Approximately 397 MB
GGUF Q8_0 size	Approximately 639 MB
GGUF F16 size	Approximately 1.2 GB

File sizes are approximate and may vary slightly by build.

Benchmark results

TinyToolRouter was evaluated on a held-out set of 231 cases covering three distinct evaluation families.

Overall performance

Table with columns: Metric, Qwen3-0.6B (base), TinyToolRouter-0.6B
Metric	Qwen3-0.6B (base)	TinyToolRouter-0.6B
Exact match	51.95%	92.21%
Valid structured output	99.13%	99.13%
Tool selection	93.51%	98.70%
Argument accuracy	51.95%	92.21%
`no_tool` precision	76.47%	97.56%
`no_tool` recall	97.50%	100.00%
`no_tool` F1	85.71%	98.77%

Exact match improved by 40.26 percentage points over the base model.

Performance by evaluation family

Table with columns: Evaluation family, Cases, Base model, Fine-tuned
Evaluation family	Cases	Base model	Fine-tuned
`seen_paraphrase`	131	22.90%	92.37%
`unseen_schema`	60	85.00%	86.67%
`no_tool`	40	97.50%	100.00%

What these results mean

Paraphrase robustness: Exact match increased from 22.90% to 92.37% on requests that use unseen phrasing for tools encountered during training.
Unseen tool schemas: Exact match increased from 85.00% to 86.67% on tools absent from the training set. This is a modest improvement, not evidence of fully general tool-schema understanding.
Abstention: The fine-tuned model correctly abstained on all 40 held-out no_tool cases, with 97.56% measured precision.

Evaluation methodology

Both models were evaluated using identical prompts and generation settings:

do_sample=False
max_new_tokens=128
Held-out evaluation set: 231 cases
Evaluation metric: exact match of the expected structured output

The benchmark separates familiar tool schemas with paraphrased requests, genuinely unseen tool schemas, and requests that should not trigger a tool call.

The reported metrics are from the safetensors model. The GGUF conversions have not been independently benchmarked, so equivalent performance across formats should not be assumed.

Quickstart

Install dependencies

bash

pip install torch transformers

Run with Transformers

python

import jsonimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Robokid1134/TinyToolRouter-0.6B" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id)model.eval() tools = [    {        "name": "get_weather",        "description": "Get the current weather for a city.",        "parameters": {            "type": "object",            "properties": {                "city": {"type": "string"}            },            "required": ["city"],        },    }] system_prompt = (    "You are a tool router. You are given a list of available tools "    "and a user request. Reply with exactly one JSON object of the form "    '{"name": "<tool_name>", "arguments": {"<argument>": "<value>"}}. '    "Use only the listed tools and their declared arguments. "    'If no listed tool applies, reply {"name": "no_tool", "arguments": {}}. '    "Output JSON only, with no additional prose.") messages = [    {"role": "system", "content": system_prompt},    {        "role": "user",        "content": json.dumps({            "tools": tools,            "request": "What's the weather in Oslo?"        }),    },] prompt = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,    enable_thinking=False,) inputs = tokenizer(prompt, return_tensors="pt") with torch.inference_mode():    output = model.generate(        **inputs,        max_new_tokens=128,        do_sample=False,    ) new_tokens = output[0][inputs["input_ids"].shape[1]:]response = tokenizer.decode(    new_tokens,    skip_special_tokens=True,) print(response)

Expected output:

json

{"name": "get_weather", "arguments": {"city": "Oslo"}}

Note: enable_thinking=False is recommended for direct routing output. The example uses the Qwen3 chat template and assumes a Transformers version that supports this argument.

GGUF: CPU inference

GGUF builds are available in the repository's gguf/ directory.

Table with columns: File, Approximate size, Intended use
File	Approximate size	Intended use
`gguf/model-Q4_K_M.gguf`	397 MB	Compact CPU inference
`gguf/model-Q8_0.gguf`	639 MB	Higher quantization fidelity
`gguf/model-f16.gguf`	1.2 GB	Full-precision weight storage

Run a GGUF model using llama.cpp:

bash

llama-cli \  -m gguf/model-Q4_K_M.gguf \  -p "<your prompt>"

The prompt should follow the model's expected chat template and include the available tool schemas and user request.

For repeated use, a dedicated prompt template or a compatible Ollama model configuration can simplify integration.

Benchmark caveat: The GGUF files were converted from the merged fine-tuned model but have not been independently re-evaluated. The benchmark figures above apply to the safetensors model.

Tool schema format

The model expects a JSON object containing a list of available tools and a natural-language request.

json

{  "tools": [    {      "name": "get_weather",      "description": "Get the current weather for a city.",      "parameters": {        "type": "object",        "properties": {          "city": {            "type": "string"          }        },        "required": ["city"]      }    }  ],  "request": "What's the weather in Oslo?"}

The schema format is JSON-Schema-style, with a name, description, and parameters object for each tool.

Training covered the following argument types:

string
integer
number
boolean
array of strings
array of numbers

Examples included required and optional arguments, along with distractor tools that should not be selected.

The model returns a proposed tool name and its arguments. It does not execute the tool or verify that the underlying action succeeds.

Training

Table with columns: Setting, Value
Setting	Value
Base checkpoint	`Qwen/Qwen3-0.6B`
Base revision	`c1899de289a04d12100db370d81485cdf75e47ca`
Fine-tuning method	LoRA using PEFT
Adapter rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	Attention and MLP projections
Epochs	3
Training steps	57
Learning rate	`1e-4`
Scheduler	Cosine
Warmup ratio	3%
Batch size	8
Gradient accumulation	4
Maximum sequence length	512 tokens
Training precision	BF16
Random seed	7
Training examples	599
Held-out examples	231
Hardware	NVIDIA A10G, 24 GB VRAM
Approximate training time	150 seconds
PyTorch	2.5.1
Transformers	4.57.6
PEFT	0.17.1
Python	3.9

The LoRA adapter was merged into the base model for deployment.

Training loss was computed on the assistant's JSON output, with prompt tokens masked from the loss.

Dataset

The dataset consists of original synthetic examples authored for this project and released under Apache-2.0.

No scraped text, chat logs, or personal data were used. Slot values are synthetic, including example email addresses, dates, and city names.

Tool coverage

Tools represented in training (9):

get_weather
send_email
create_calendar_event
search_web
convert_currency
set_timer
translate_text
filter_items
tag_note

Unseen tools reserved for evaluation (5):

play_music
get_stock_price
book_restaurant
draw_chart
create_reminder

Data separation

The training and held-out sets use disjoint templates, value pools, and example IDs.

The unseen_schema evaluation family contains tools absent from training entirely. The training set also includes abstention examples, while the held-out no_tool requests are disjoint from those training examples.

These design choices help test whether the model learned useful routing patterns rather than simply memorizing identical examples. They do not eliminate every possible source of dataset bias or leakage.

Limitations

English only: The model was trained and evaluated on English-language examples.
Limited tool coverage: Training covers nine tool schemas, with five additional schemas used for evaluation.
Synthetic data: Performance on real-world agent requests may differ from the reported benchmark.
Modest unseen-schema gains: The improvement on unseen schemas is only 1.67 percentage points.
Limited argument extraction: Arguments often appear explicitly in the request. The evaluation primarily tests tool selection, argument correctness, and structured output rather than unrestricted information extraction.
Small evaluation set: Results are based on 231 held-out examples. Small differences may reflect only a few cases.
Single random seed: Multi-seed variance has not been measured.
Quantized models not re-benchmarked: The reported metrics do not establish equivalent performance for the GGUF builds.
Context length: Training used a maximum sequence length of 512 tokens. Long tool lists may exceed the model's familiar input distribution or be truncated by an inference pipeline.

Intended use

TinyToolRouter-0.6B is intended for:

Local AI agents running on CPU-only or low-VRAM machines.
Lightweight tool-selection front ends for larger language models.
Routing between a predefined set of application tools.
Explicitly declining requests that do not match any available tool.
Research into compact language models, tool calling, and structured generation.
Reproducible experiments comparing a small base model against a narrowly fine-tuned router.

Out of scope

This model is not a general-purpose assistant. It is not intended to independently answer general questions, execute tools, or make safety-critical decisions.

Do not rely on its output alone to authorize sensitive actions or execute shell commands. The calling application should validate the selected tool, enforce an allowlist, validate arguments against the actual schema, and require appropriate authorization before executing an action.

Citation

If you use this model in research or another project, you can cite it as follows:

bibtex

@misc{tinytoolrouter06b,  title  = {TinyToolRouter-0.6B: A CPU-First, Schema-Strict Tool Router},  author = {Robokid1134},  year   = {2026},  note   = {LoRA fine-tune of Qwen/Qwen3-0.6B},  url    = {https://huggingface.co/Robokid1134/TinyToolRouter-0.6B}}

Attribution and license

TinyToolRouter-0.6B is fine-tuned from Qwen/Qwen3-0.6B, which is released under the Apache-2.0 license.

This derivative model is distributed under Apache-2.0, with the applicable upstream license and copyright notices preserved. The project's original code and generated dataset are also released under Apache-2.0.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Robokid1134

Model Tree

Base

Qwen/Qwen3-0.6B

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Key features

Small footprint: 0.6B parameters, suitable for CPU-first inference.
Schema-aware routing: Selects from tool definitions supplied at inference time.
Structured output: Produces a single JSON object representing a proposed tool call.
Explicit abstention: Supports a dedicated no_tool response.
Paraphrase robustness: Evaluated on requests phrased differently from the training examples.
Unseen-schema evaluation: Tested on tools absent from the training set.
Multiple formats: Safetensors and GGUF builds are available in this repository.

Model details

Table with columns: Property, Value
Property	Value
Model	TinyToolRouter-0.6B
Base model	Qwen/Qwen3-0.6B
Parameters	0.6B
Fine-tuning	Supervised fine-tuning with LoRA (PEFT)
Deployment	Merged fine-tuned weights
Primary language	English
License	Apache-2.0
Intended hardware	CPU-only and low-VRAM systems
Safetensors size	Approximately 1.2 GB
GGUF Q4_K_M size	Approximately 397 MB
GGUF Q8_0 size	Approximately 639 MB
GGUF F16 size	Approximately 1.2 GB

File sizes are approximate and may vary slightly by build.

Benchmark results

TinyToolRouter was evaluated on a held-out set of 231 cases covering three distinct evaluation families.

Overall performance

Table with columns: Metric, Qwen3-0.6B (base), TinyToolRouter-0.6B
Metric	Qwen3-0.6B (base)	TinyToolRouter-0.6B
Exact match	51.95%	92.21%
Valid structured output	99.13%	99.13%
Tool selection	93.51%	98.70%
Argument accuracy	51.95%	92.21%
`no_tool` precision	76.47%	97.56%
`no_tool` recall	97.50%	100.00%
`no_tool` F1	85.71%	98.77%

Exact match improved by 40.26 percentage points over the base model.

Performance by evaluation family

Table with columns: Evaluation family, Cases, Base model, Fine-tuned
Evaluation family	Cases	Base model	Fine-tuned
`seen_paraphrase`	131	22.90%	92.37%
`unseen_schema`	60	85.00%	86.67%
`no_tool`	40	97.50%	100.00%

What these results mean

Paraphrase robustness: Exact match increased from 22.90% to 92.37% on requests that use unseen phrasing for tools encountered during training.
Unseen tool schemas: Exact match increased from 85.00% to 86.67% on tools absent from the training set. This is a modest improvement, not evidence of fully general tool-schema understanding.
Abstention: The fine-tuned model correctly abstained on all 40 held-out no_tool cases, with 97.56% measured precision.

Evaluation methodology

Both models were evaluated using identical prompts and generation settings:

do_sample=False
max_new_tokens=128
Held-out evaluation set: 231 cases
Evaluation metric: exact match of the expected structured output

The benchmark separates familiar tool schemas with paraphrased requests, genuinely unseen tool schemas, and requests that should not trigger a tool call.

The reported metrics are from the safetensors model. The GGUF conversions have not been independently benchmarked, so equivalent performance across formats should not be assumed.

Quickstart

Install dependencies

bash

pip install torch transformers

Run with Transformers

python

import jsonimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Robokid1134/TinyToolRouter-0.6B" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id)model.eval() tools = [    {        "name": "get_weather",        "description": "Get the current weather for a city.",        "parameters": {            "type": "object",            "properties": {                "city": {"type": "string"}            },            "required": ["city"],        },    }] system_prompt = (    "You are a tool router. You are given a list of available tools "    "and a user request. Reply with exactly one JSON object of the form "    '{"name": "<tool_name>", "arguments": {"<argument>": "<value>"}}. '    "Use only the listed tools and their declared arguments. "    'If no listed tool applies, reply {"name": "no_tool", "arguments": {}}. '    "Output JSON only, with no additional prose.") messages = [    {"role": "system", "content": system_prompt},    {        "role": "user",        "content": json.dumps({            "tools": tools,            "request": "What's the weather in Oslo?"        }),    },] prompt = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,    enable_thinking=False,) inputs = tokenizer(prompt, return_tensors="pt") with torch.inference_mode():    output = model.generate(        **inputs,        max_new_tokens=128,        do_sample=False,    ) new_tokens = output[0][inputs["input_ids"].shape[1]:]response = tokenizer.decode(    new_tokens,    skip_special_tokens=True,) print(response)

import jsonimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Robokid1134/TinyToolRouter-0.6B" tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id)model.eval() tools = [    {        "name": "get_weather",        "description": "Get the current weather for a city.",        "parameters": {            "type": "object",            "properties": {                "city": {"type": "string"}            },            "required": ["city"],        },    }] system_prompt = (    "You are a tool router. You are given a list of available tools "    "and a user request. Reply with exactly one JSON object of the form "    '{"name": "<tool_name>", "arguments": {"<argument>": "<value>"}}. '    "Use only the listed tools and their declared arguments. "    'If no listed tool applies, reply {"name": "no_tool", "arguments": {}}. '    "Output JSON only, with no additional prose.") messages = [    {"role": "system", "content": system_prompt},    {        "role": "user",        "content": json.dumps({            "tools": tools,            "request": "What's the weather in Oslo?"        }),    },] prompt = tokenizer.apply_chat_template(    messages,    tokenize=False,    add_generation_prompt=True,    enable_thinking=False,) inputs = tokenizer(prompt, return_tensors="pt") with torch.inference_mode():    output = model.generate(        **inputs,        max_new_tokens=128,        do_sample=False,    ) new_tokens = output[0][inputs["input_ids"].shape[1]:]response = tokenizer.decode(    new_tokens,    skip_special_tokens=True,) print(response)

Expected output:

json

{"name": "get_weather", "arguments": {"city": "Oslo"}}

{"name": "get_weather", "arguments": {"city": "Oslo"}}

Note: enable_thinking=False is recommended for direct routing output. The example uses the Qwen3 chat template and assumes a Transformers version that supports this argument.

GGUF: CPU inference

GGUF builds are available in the repository's gguf/ directory.

Table with columns: File, Approximate size, Intended use
File	Approximate size	Intended use
`gguf/model-Q4_K_M.gguf`	397 MB	Compact CPU inference
`gguf/model-Q8_0.gguf`	639 MB	Higher quantization fidelity
`gguf/model-f16.gguf`	1.2 GB	Full-precision weight storage

Run a GGUF model using llama.cpp:

bash

llama-cli \  -m gguf/model-Q4_K_M.gguf \  -p "<your prompt>"

llama-cli \  -m gguf/model-Q4_K_M.gguf \  -p "<your prompt>"

The prompt should follow the model's expected chat template and include the available tool schemas and user request.

For repeated use, a dedicated prompt template or a compatible Ollama model configuration can simplify integration.

Benchmark caveat: The GGUF files were converted from the merged fine-tuned model but have not been independently re-evaluated. The benchmark figures above apply to the safetensors model.

Tool schema format

The model expects a JSON object containing a list of available tools and a natural-language request.

json

{  "tools": [    {      "name": "get_weather",      "description": "Get the current weather for a city.",      "parameters": {        "type": "object",        "properties": {          "city": {            "type": "string"          }        },        "required": ["city"]      }    }  ],  "request": "What's the weather in Oslo?"}

{  "tools": [    {      "name": "get_weather",      "description": "Get the current weather for a city.",      "parameters": {        "type": "object",        "properties": {          "city": {            "type": "string"          }        },        "required": ["city"]      }    }  ],  "request": "What's the weather in Oslo?"}

The schema format is JSON-Schema-style, with a name, description, and parameters object for each tool.

Training covered the following argument types:

string
integer
number
boolean
array of strings
array of numbers

Examples included required and optional arguments, along with distractor tools that should not be selected.

The model returns a proposed tool name and its arguments. It does not execute the tool or verify that the underlying action succeeds.

Training

Table with columns: Setting, Value
Setting	Value
Base checkpoint	`Qwen/Qwen3-0.6B`
Base revision	`c1899de289a04d12100db370d81485cdf75e47ca`
Fine-tuning method	LoRA using PEFT
Adapter rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	Attention and MLP projections
Epochs	3
Training steps	57
Learning rate	`1e-4`
Scheduler	Cosine
Warmup ratio	3%
Batch size	8
Gradient accumulation	4
Maximum sequence length	512 tokens
Training precision	BF16
Random seed	7
Training examples	599
Held-out examples	231
Hardware	NVIDIA A10G, 24 GB VRAM
Approximate training time	150 seconds
PyTorch	2.5.1
Transformers	4.57.6
PEFT	0.17.1
Python	3.9

The LoRA adapter was merged into the base model for deployment.

Training loss was computed on the assistant's JSON output, with prompt tokens masked from the loss.

Dataset

The dataset consists of original synthetic examples authored for this project and released under Apache-2.0.

No scraped text, chat logs, or personal data were used. Slot values are synthetic, including example email addresses, dates, and city names.

Tool coverage

Tools represented in training (9):

get_weather
send_email
create_calendar_event
search_web
convert_currency
set_timer
translate_text
filter_items
tag_note

Unseen tools reserved for evaluation (5):

play_music
get_stock_price
book_restaurant
draw_chart
create_reminder

Data separation

The training and held-out sets use disjoint templates, value pools, and example IDs.

Limitations

English only: The model was trained and evaluated on English-language examples.
Limited tool coverage: Training covers nine tool schemas, with five additional schemas used for evaluation.
Synthetic data: Performance on real-world agent requests may differ from the reported benchmark.
Modest unseen-schema gains: The improvement on unseen schemas is only 1.67 percentage points.
Limited argument extraction: Arguments often appear explicitly in the request. The evaluation primarily tests tool selection, argument correctness, and structured output rather than unrestricted information extraction.
Small evaluation set: Results are based on 231 held-out examples. Small differences may reflect only a few cases.
Single random seed: Multi-seed variance has not been measured.
Quantized models not re-benchmarked: The reported metrics do not establish equivalent performance for the GGUF builds.
Context length: Training used a maximum sequence length of 512 tokens. Long tool lists may exceed the model's familiar input distribution or be truncated by an inference pipeline.

Intended use

TinyToolRouter-0.6B is intended for:

Local AI agents running on CPU-only or low-VRAM machines.
Lightweight tool-selection front ends for larger language models.
Routing between a predefined set of application tools.
Explicitly declining requests that do not match any available tool.
Research into compact language models, tool calling, and structured generation.
Reproducible experiments comparing a small base model against a narrowly fine-tuned router.

Out of scope

This model is not a general-purpose assistant. It is not intended to independently answer general questions, execute tools, or make safety-critical decisions.

Citation

If you use this model in research or another project, you can cite it as follows:

bibtex

@misc{tinytoolrouter06b,  title  = {TinyToolRouter-0.6B: A CPU-First, Schema-Strict Tool Router},  author = {Robokid1134},  year   = {2026},  note   = {LoRA fine-tune of Qwen/Qwen3-0.6B},  url    = {https://huggingface.co/Robokid1134/TinyToolRouter-0.6B}}

@misc{tinytoolrouter06b,  title  = {TinyToolRouter-0.6B: A CPU-First, Schema-Strict Tool Router},  author = {Robokid1134},  year   = {2026},  note   = {LoRA fine-tune of Qwen/Qwen3-0.6B},  url    = {https://huggingface.co/Robokid1134/TinyToolRouter-0.6B}}

Attribution and license

TinyToolRouter-0.6B is fine-tuned from Qwen/Qwen3-0.6B, which is released under the Apache-2.0 license.