Model Description
SakThai Context 0.5B Tools is a prompt-masked supervised fine-tune of Qwen/Qwen2.5-0.5B-Instruct focused on reliable tool/function calling in conversational agents. The model is trained to select the correct tool, generate valid JSON-style arguments, and avoid degenerate outputs. It is optimized for edge deployment and can run on consumer hardware with ~1 GB RAM.
Key points:
- Base:
Qwen/Qwen2.5-0.5B-Instruct
- Training: prompt-masked SFT on tool-calling traces from
Nanthasit/sakthai-combined-v7
- Primary use: lightweight agents, on-device assistants, Raspberry Pi / edge deployments
- License: Apache-2.0
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Nanthasit/sakthai-context-0.5b-tools"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather",
"parameters": {
"type": "object",
"properties": {"location": {"type": "string"}},
"required": ["location"],
},
}
}
]
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What's the weather in Tokyo?"},
]
text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.01, top_p=0.9)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
Quick Start — llama.cpp / Ollama
# Convert with llama.cpp and run locally
llama-quantize ./sakthai-context-0.5b-tools-f16.gguf ./model-q4_k_m.gguf Q4_K_M
ollama create sakthai-context-0.5b-tools -f Modelfile
ollama run sakthai-context-0.5b-tools
Usage notes
- For tool calling, always use
apply_chat_template(..., tools=tools, tokenize=False, add_generation_prompt=True) so the model receives the proper <tools> block.
- If you want stricter outputs, reduce
temperature further, e.g. 0.0.
- For CPU-only inference, set
device_map="cpu"; GPU/MPS/CPU auto-detection works with device_map="auto".
Architecture & Config
Table with columns: Field, Value| Field | Value |
|---|
| Architecture | Qwen2ForCausalLM |
| Model type | qwen2 |
| Vocab size | 151936 |
| Hidden size | 896 |
| Layers | 24 |
| Attention heads | 14 |
Benchmarks
Table with columns: Metric, Value, Verified| Metric | Value | Verified |
|---|
| Selection Accuracy | 91.2% | true |
| Arguments Accuracy | 45.7% | true |
| Strict Accuracy | 45.7% | true |
| Held-Out Tool Accuracy | 87.8% | true |
| Degenerate Outputs | 0% | true |
Evidence: .eval_results/sakthai-bench-v2.yaml in repo.
Limitations
- 0.5B parameter scale limits reasoning depth; arguments accuracy is lower than selection accuracy.
- Tool schema adherence degrades on nested arguments and long context traces.
- Current weights are merged; if you need the unmerged adapter, use
Nanthasit/sakthai-context-0.5b-tools-sft or Nanthasit/sakthai-context-0.5b-tools-sft-v2.
Citation
If you use this model, please cite the SakThai model family and benchmark:
@misc{sakthai2025context05btools,
title = {SakThai Context 0.5B Tools},
author = {Nanthasit},
year = {2026},
url = {https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools}
}
SakThai Family
Download counts and sizes were verified from the Hub API at upload time.