Nanthasit

sakthai-context-0.5b-tools

README

License: apache-2.0

Model Description

SakThai Context 0.5B Tools is a prompt-masked supervised fine-tune of Qwen/Qwen2.5-0.5B-Instruct focused on reliable tool/function calling in conversational agents. The model is trained to select the correct tool, generate valid JSON-style arguments, and avoid degenerate outputs. It is optimized for edge deployment and can run on consumer hardware with ~1 GB RAM.

Key points:

Base: Qwen/Qwen2.5-0.5B-Instruct
Training: prompt-masked SFT on tool-calling traces from Nanthasit/sakthai-combined-v7
Primary use: lightweight agents, on-device assistants, Raspberry Pi / edge deployments
License: Apache-2.0

Quick Start — Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Nanthasit/sakthai-context-0.5b-tools"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto") tools = [    {        "type": "function",        "function": {            "name": "get_weather",            "description": "Get current weather",            "parameters": {                "type": "object",                "properties": {"location": {"type": "string"}},                "required": ["location"],            },        }    }] messages = [    {"role": "system", "content": "You are a helpful assistant."},    {"role": "user", "content": "What's the weather in Tokyo?"},] text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)inputs = tokenizer([text], return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.01, top_p=0.9)response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)print(response)

Quick Start — llama.cpp / Ollama

bash

# Convert with llama.cpp and run locallyllama-quantize ./sakthai-context-0.5b-tools-f16.gguf ./model-q4_k_m.gguf Q4_K_Mollama create sakthai-context-0.5b-tools -f Modelfileollama run sakthai-context-0.5b-tools

Usage notes

For tool calling, always use apply_chat_template(..., tools=tools, tokenize=False, add_generation_prompt=True) so the model receives the proper <tools> block.
If you want stricter outputs, reduce temperature further, e.g. 0.0.
For CPU-only inference, set device_map="cpu"; GPU/MPS/CPU auto-detection works with device_map="auto".

Architecture & Config

Table with columns: Field, Value
Field	Value
Architecture	`Qwen2ForCausalLM`
Model type	`qwen2`
Vocab size	`151936`
Hidden size	`896`
Layers	`24`
Attention heads	`14`

Benchmarks

Table with columns: Metric, Value, Verified
Metric	Value	Verified
Selection Accuracy	91.2%	true
Arguments Accuracy	45.7%	true
Strict Accuracy	45.7%	true
Held-Out Tool Accuracy	87.8%	true
Degenerate Outputs	0%	true

Evidence: .eval_results/sakthai-bench-v2.yaml in repo.

Limitations

0.5B parameter scale limits reasoning depth; arguments accuracy is lower than selection accuracy.
Tool schema adherence degrades on nested arguments and long context traces.
Current weights are merged; if you need the unmerged adapter, use Nanthasit/sakthai-context-0.5b-tools-sft or Nanthasit/sakthai-context-0.5b-tools-sft-v2.

Citation

If you use this model, please cite the SakThai model family and benchmark:

bibtex

@misc{sakthai2025context05btools,  title  = {SakThai Context 0.5B Tools},  author = {Nanthasit},  year   = {2026},  url    = {https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools}}

SakThai Family

Table with columns: Repo, Downloads, Size, Pipeline
Repo	Downloads	Size	Pipeline
Nanthasit/sakthai-context-1.5b-merged	1855	~4.07 GB	text-generation
Nanthasit/sakthai-context-0.5b-merged	1692	~1.39 GB	text-generation
Nanthasit/sakthai-context-7b-merged	1024

Download counts and sizes were verified from the Hub API at upload time.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Nanthasit

Model Tree

Base

Qwen/Qwen2.5-0.5B-Instruct

Fine-tuned

this model

Input Modalities

Text

Output Modalities