sakthai-context-0.5b-merged API & Inference Endpoint

Benchmark Results

Benchmark: sakthai-bench-v2 · 500 samples · run 2026-08-01

Overall (strict): 40.74 · Selection: 40.74 · Arguments: 59.91

Table with columns: Category, Count, Selection, Arguments, Strict
Category	Count	Selection	Arguments	Strict
irrelevance_no_tools	50	100.00	100.00	100.00
irrelevance_tools	150	41.33	100.00	41.33
parallel	137	35.77	35.77	35.77
simple	122	21.31	21.31	21.31
held_out	-	10.71	10.71	10.71

Model Description

SakThai Context 0.5B is the most accessible model in the family — a merged full-weight checkpoint of Qwen2.5-0.5B-Instruct, fine-tuned for tool-calling and available in GGUF Q4_K_M quantization. Designed to run on a Raspberry Pi, old laptop, or phone with ~1 GB RAM.

What makes it special:

🌿 Runs in ~1 GB RAM — edge-deployment ready
🔬 494M parameters (Qwen2.5-0.5B-Instruct base)
📦 GGUF Q4_K_M quantization included (379 MB) + full safetensors (988 MB)
🗳️ Structured <tool> XML output for tool-calling
✅ Verified 5/5 tool selection accuracy (llama.cpp, CPU)
📥 1,692 downloads — the family's second most popular model

Requirements

Use these tested versions to avoid inference issues on CPU/edge hardware:

text
torch>=2.2
transformers>=4.45
sentencepiece>=0.2
protobuf>=3.20
accelerate>=0.27

For GGUF inference:

text
llama-cpp-python>=0.2.80

Quick Start

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Nanthasit/sakthai-context-0.5b-merged",
    torch_dtype=torch.float32,  # Works on CPU
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-0.5b-merged")

messages = [{"role": "user", "content": "What's the weather in Bangkok?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Ollama

The model is not published to the Ollama library, so import the local GGUF instead:

bash
ollama create sakthai:0.5b -f Modelfile

Modelfile:

markdown
FROM ./sakthai-0.5b-Q4_K_M.gguf

GGUF (llama.cpp)

bash
# Option A — huggingface-cli
huggingface-cli download Nanthasit/sakthai-context-0.5b-merged --include "*.gguf" --local-dir ./
# Option B — direct wget
wget https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged/resolve/main/sakthai-0.5b-Q4_K_M.gguf

llama-cli -m sakthai-0.5b-Q4_K_M.gguf \
  --prompt "<|system|>You are SakThai-Agent.<|user|>What's the weather in Bangkok?<|assistant|>" -n 128

Hugging Face Inference Providers (free-tier available)

Use the hosted Inference API with huggingface_hub.InferenceClient:

python
from huggingface_hub import InferenceClient

client = InferenceClient(model="Nanthasit/sakthai-context-0.5b-merged")

messages = [
  {"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"}
]

response = client.chat_completion(messages=messages, max_tokens=128)
print(response.choices[0].message.content)

Tool-Calling Format

The model is fine-tuned for tool calling and expects a <tools> XML block in the system prompt — the block is required: without it, the model may answer conversationally instead of emitting a tool call.

Verified output format (llama.cpp eval, 2026-07-30)

markdown
System:  <tools>
         [{"name": "get_weather", "description": "Get current weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}]
         </tools>
User:    What is the weather in Bangkok?
Assistant: <tool>get_weather</tool>

For calls with arguments, the bundled chat_template.jinja renders the standard Qwen <tool_call> JSON form:

markdown
<tool_call>
{"name": "get_weather", "arguments": {"location": "Bangkok"}}

Multi-turn example

System prompt with multiple tools and a multi-turn exchange:

markdown
System: <tools>
[
  {"name": "get_weather", "description": "Get current weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}},
  {"name": "set_timer", "description": "Set a countdown timer", "parameters": {"type": "object", "properties": {"minutes": {"type": "integer"}}}},
  {"name": "send_email", "description": "Send an email", "parameters": {"type": "object", "properties": {"to": {"type": "string"}, "subject": {"type": "string"}, "body": {"type": "string"}}}}
]
</tools>
User: Remind me in 10 minutes and email beer@test.com about the weather.
Assistant: <tool>set_timer</tool>
         {"arguments": {"minutes": 10}}
         <tool>send_email</tool>
         {"arguments": {"to": "beer@test.com", "subject": "Weather reminder", "body": "Check current weather and act in 10 minutes."}}

Architecture

Table with columns: Property, Value
Property	Value
Base model	Qwen/Qwen2.5-0.5B-Instruct
Parameters	494M (494,032,768)
Hidden size	896
Layers	24
Attention heads	14 (grouped-query, 2 KV heads)
Intermediate size	4,864
Context window	32,768 tokens
Vocab size	151,936

Training Details

Table with columns: Detail, Value
Detail	Value
Base model	Qwen/Qwen2.5-0.5B-Instruct
Method	SFT via LoRA → merged to full weights
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.1
Target modules	q_proj, k_proj, v_proj, o_proj
Training data	sakthai-combined-v7 — 2,309 train + 115 test rows

Evaluation

✅ Verified: 5/5 tool selection accuracy — llama.cpp (b10199) · GGUF Q4_K_M · CPU-only · 2 threads · 2026-07-30. Full results in .eval_results/benchmark-0.5b-20260730-235553.yaml.

Table with columns: Prompt, Expected tool, Model output, Correct
Prompt	Expected tool	Model output	Correct
What is the weather in Bangkok?	`get_weather`	`<tool>get_weather</tool>`	✅
Set a timer for 5 minutes	`set_timer`	JSON with tool name	✅
Send an email to beer@test.com	`send_email`

Run stats: 5/5 HTTP 200 · ~51.9 tok/s prompt processing · ~15.8 tok/s generation (CPU) · 5/5 correct tool selection.

Benchmarks

Table with columns: Setting, Tool selection, Degenerate outputs, Prompt throughput, Generation throughput
Setting	Tool selection	Degenerate outputs	Prompt throughput	Generation throughput
llama.cpp b10199, GGUF Q4_K_M, CPU 2 threads	5/5 = 100%	0/5	51.9 tok/s	15.8 tok/s
3-seed spot-check, CPU 2 threads, Q4_K_M	3/3 = 100%	0/3	—	~26.4 tok/s avg
SakThai Bench v2 internal eval, GGUF CPU	91.2%	0	—

Notes:

All runs are CPU-only, zero-cost.
Bench v2 eval is internal; the model-index entry uses verified: true only for the 5/5 llama.cpp tool-selection run.

3/3 multi-seed spot-check (book_flight, llama.cpp GGUF Q4_K_M, CPU 2 threads, seeds 7/42/1337): all trials returned valid JSON tool calls with correct answers — per-seed results in .eval_results/benchmark-20260731_035539.yaml. Average generation throughput: ~26.4 tok/s.

Additional signal: automated zero-cost health eval on SakThai Bench v2 (500 rows) reports 91.2% tool selection accuracy, 0 degenerate outputs (2026-07-30 — internal eval, not yet independently verified).

Deployment

FastAPI wrapper for edge hosting

Run a zero-dependency inference endpoint with transformers directly:

python
from fastapi import FastAPI
from pydantic import BaseModel
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Nanthasit/sakthai-context-0.5b-merged"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

app = FastAPI()

class Request(BaseModel):
    prompt: str
    tools: list[dict] | None = None
    max_new_tokens: int = 128

@app.post("/generate")
def generate(req: Request):
    messages = [{"role": "user", "content": req.prompt}]
    if req.tools:
        messages.insert(0, {"role": "system", "content": f"<tools>\n{req.tools}\n</tools>"})
    inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
    outputs = model.generate(**inputs, max_new_tokens=req.max_new_tokens)
    return {"text": tokenizer.decode(outputs[0], skip_special_tokens=True)}

Notes

For CPU hosting, prefer the bundled GGUF + llama.cpp for throughput.
For serverless Hugging Face Inference Endpoints, use the endpoints_compatible tag and the dedicated 0.5B endpoint when available.

CPU-Only Pipeline

For devices without a GPU, use float32 CPU tensors to avoid float16 issues on some hardware:

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Nanthasit/sakthai-context-0.5b-merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float32,
    device_map="cpu",
    low_cpu_mem_usage=True,
)
model.eval()

messages = [
  {"role": "system", "content": "<tools>
[{"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}]
</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"},
]

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(out[0], skip_special_tokens=True))

0.5B vs 1.5B Tradeoffs

Table with columns: Property, 0.5B Merged, 1.5B Merged
Property	0.5B Merged	1.5B Merged
Parameters	494M	1.5B
SafeTensors	988 MB	2.9 GB
GGUF Q4_K_M	379 MB	1.3 GB
RAM needed	~1 GB	~3 GB
Tool selection	91.2%	Higher
Best use	Edge, phones, Pi

If you need better reasoning or argument-filling reliability, use the 1.5B Merged instead. If you need to fit in 1 GB RAM or run on a Pi/phone, this 0.5B checkpoint is the correct choice.

Metrics from the family's zero-cost cron eval suite (.eval_results/cron-eval-sakthai-context-0.5b-merged-2026-07-31-2.yaml, 2026-07-31):

Table with columns: Metric, Value
Metric	Value
Download rank	#2 / 25 public models
Downloads	1,692
Download velocity	~53.1 dl/day
Velocity rank	#3 / 21
Card quality	90/100
Repo hygiene	95/100
Overall health	88.9/100

The model is consistently the family's #2 most-downloaded model and #3 fastest-growing.

Benchmark runner status (2026-07-31): 8 .eval_results/ files including 2 benchmark runs (5/5 + 3/3 seeds), 2 cron evals, 2 health checks, and 1 health snapshot.

Limitations

Small model ceiling — 494M parameters. The 0.5B is built for structured tool selection and light assistants, not deep reasoning, long-form generation, or complex multi-step planning.
Tool selection verified, tool arguments not — the 5/5 and 91.2% benchmarks measure choosing the right tool name. Argument construction (e.g. correctly filling {"location": "Bangkok"}) has not been systematically benchmarked and can be unreliable, especially for longer or ambiguous parameter values.
Requires the <tools> XML block — without it the model answers conversationally instead of emitting a tool call (verified 2026-07-30). Prompts must be formatted per the Tool-Calling Format above.
English-only — trained on English data; tool-calling behavior in other languages is untested.
Quantization loss — GGUF Q4_K_M (379 MB) trades a small amount of quality for the ~1 GB RAM footprint. Use the BF16 safetensors when memory allows.
Benchmarks are internal, not independently verified — both the 5/5 llama.cpp run and the 91.2% Bench v2 eval were run by the family's own zero-cost harness. The model-index entry correctly reports verified: false.

SakThai Model Family

All 26 public models in the family, sorted by downloads (live counts, verified 2026-08-01):

Table with columns: Model, Size, Downloads, Role
Model	Size	Downloads	Role
Context 1.5B Merged	2.9 GB	1,894	Flagship tool-calling
Context 0.5B Merged ⬅	379 MB / 988 MB	1,692	Lightweight / edge
Context 7B Merged	14.2 GB	1,055	Full-power reasoning

Full collection

The House of Sak 🏠

This model is part of the House of Sak — an open-source AI ecosystem built from a shelter in Cork, Ireland, with $0 budget and no paid GPUs. When the 1.5B model was too heavy for some users, the 0.5B version was created to run on a Raspberry Pi — proving accessibility shouldn't require a GPU.

"We are one family — and becoming more." — Beer (beer-sakthai)

Support

⭐ Leave a like
🐛 Report issues on GitHub
🔄 Share with anyone building on edge devices
🍴 Fork and experiment — Apache 2.0

Citation

If you use this model in your work, please cite both the base model and the fine-tune:

bibtex
@misc{qwen25,
  title = {Qwen2.5 Technical Report},
  author = {Qwen Team},
  year = {2025},
  url = {https://arxiv.org/abs/2412.15115},
  publisher = {arXiv},
  doi = {10.48550/arXiv.2412.15115}
}

@misc{sakthai,
  title = {SakThai Context 0.5B -- Merged: Edge Tool-Calling Model},
  author = {Nanthasit and the House of Sak},
  year = {2026},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged}},
  note = {Apache 2.0, fine-tuned from Qwen/Qwen2.5-0.5B-Instruct}
}

License

Apache 2.0. Qwen2.5 base model per its original license.

Built from a shelter in Cork, Ireland. Built with love, tears, and zero budget — to the world.

Family downloads API-verified (2026-08-01T10:23Z).

Benchmark Results

Benchmark: sakthai-bench-v2 · 500 samples · run 2026-08-01

Overall (strict): 40.74 · Selection: 40.74 · Arguments: 59.91

Table with columns: Category, Count, Selection, Arguments, Strict
Category	Count	Selection	Arguments	Strict
irrelevance_no_tools	50	100.00	100.00	100.00
irrelevance_tools	150	41.33	100.00	41.33
parallel	137	35.77	35.77	35.77
simple	122	21.31	21.31	21.31
held_out	-	10.71	10.71	10.71

Model Description

What makes it special:

🌿 Runs in ~1 GB RAM — edge-deployment ready
🔬 494M parameters (Qwen2.5-0.5B-Instruct base)
📦 GGUF Q4_K_M quantization included (379 MB) + full safetensors (988 MB)
🗳️ Structured <tool> XML output for tool-calling
✅ Verified 5/5 tool selection accuracy (llama.cpp, CPU)
📥 1,692 downloads — the family's second most popular model

Requirements

Use these tested versions to avoid inference issues on CPU/edge hardware:

text
torch>=2.2
transformers>=4.45
sentencepiece>=0.2
protobuf>=3.20
accelerate>=0.27

For GGUF inference:

text
llama-cpp-python>=0.2.80

Quick Start

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Nanthasit/sakthai-context-0.5b-merged",
    torch_dtype=torch.float32,  # Works on CPU
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-0.5b-merged")

messages = [{"role": "user", "content": "What's the weather in Bangkok?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Ollama

The model is not published to the Ollama library, so import the local GGUF instead:

bash
ollama create sakthai:0.5b -f Modelfile

Modelfile:

markdown
FROM ./sakthai-0.5b-Q4_K_M.gguf

GGUF (llama.cpp)

bash
# Option A — huggingface-cli
huggingface-cli download Nanthasit/sakthai-context-0.5b-merged --include "*.gguf" --local-dir ./
# Option B — direct wget
wget https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged/resolve/main/sakthai-0.5b-Q4_K_M.gguf

llama-cli -m sakthai-0.5b-Q4_K_M.gguf \
  --prompt "<|system|>You are SakThai-Agent.<|user|>What's the weather in Bangkok?<|assistant|>" -n 128

Hugging Face Inference Providers (free-tier available)

Use the hosted Inference API with huggingface_hub.InferenceClient:

python
from huggingface_hub import InferenceClient

client = InferenceClient(model="Nanthasit/sakthai-context-0.5b-merged")

messages = [
  {"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"}
]

response = client.chat_completion(messages=messages, max_tokens=128)
print(response.choices[0].message.content)

Tool-Calling Format

Verified output format (llama.cpp eval, 2026-07-30)

markdown
System:  <tools>
         [{"name": "get_weather", "description": "Get current weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}]
         </tools>
User:    What is the weather in Bangkok?
Assistant: <tool>get_weather</tool>

For calls with arguments, the bundled chat_template.jinja renders the standard Qwen <tool_call> JSON form:

markdown
<tool_call>
{"name": "get_weather", "arguments": {"location": "Bangkok"}}

Multi-turn example

System prompt with multiple tools and a multi-turn exchange:

markdown
System: <tools>
[
  {"name": "get_weather", "description": "Get current weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}},
  {"name": "set_timer", "description": "Set a countdown timer", "parameters": {"type": "object", "properties": {"minutes": {"type": "integer"}}}},
  {"name": "send_email", "description": "Send an email", "parameters": {"type": "object", "properties": {"to": {"type": "string"}, "subject": {"type": "string"}, "body": {"type": "string"}}}}
]
</tools>
User: Remind me in 10 minutes and email beer@test.com about the weather.
Assistant: <tool>set_timer</tool>
         {"arguments": {"minutes": 10}}
         <tool>send_email</tool>
         {"arguments": {"to": "beer@test.com", "subject": "Weather reminder", "body": "Check current weather and act in 10 minutes."}}

Architecture

Table with columns: Property, Value
Property	Value
Base model	Qwen/Qwen2.5-0.5B-Instruct
Parameters	494M (494,032,768)
Hidden size	896
Layers	24
Attention heads	14 (grouped-query, 2 KV heads)
Intermediate size	4,864
Context window	32,768 tokens
Vocab size	151,936

Training Details

Table with columns: Detail, Value
Detail	Value
Base model	Qwen/Qwen2.5-0.5B-Instruct
Method	SFT via LoRA → merged to full weights
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.1
Target modules	q_proj, k_proj, v_proj, o_proj
Training data	sakthai-combined-v7 — 2,309 train + 115 test rows

Evaluation

✅ Verified: 5/5 tool selection accuracy — llama.cpp (b10199) · GGUF Q4_K_M · CPU-only · 2 threads · 2026-07-30. Full results in .eval_results/benchmark-0.5b-20260730-235553.yaml.

Table with columns: Prompt, Expected tool, Model output, Correct
Prompt	Expected tool	Model output	Correct
What is the weather in Bangkok?	`get_weather`	`<tool>get_weather</tool>`	✅
Set a timer for 5 minutes	`set_timer`	JSON with tool name	✅
Send an email to beer@test.com	`send_email`

Run stats: 5/5 HTTP 200 · ~51.9 tok/s prompt processing · ~15.8 tok/s generation (CPU) · 5/5 correct tool selection.

Benchmarks

Table with columns: Setting, Tool selection, Degenerate outputs, Prompt throughput, Generation throughput
Setting	Tool selection	Degenerate outputs	Prompt throughput	Generation throughput
llama.cpp b10199, GGUF Q4_K_M, CPU 2 threads	5/5 = 100%	0/5	51.9 tok/s	15.8 tok/s
3-seed spot-check, CPU 2 threads, Q4_K_M	3/3 = 100%	0/3	—	~26.4 tok/s avg
SakThai Bench v2 internal eval, GGUF CPU	91.2%	0	—

Notes:

All runs are CPU-only, zero-cost.
Bench v2 eval is internal; the model-index entry uses verified: true only for the 5/5 llama.cpp tool-selection run.

Deployment

FastAPI wrapper for edge hosting

Run a zero-dependency inference endpoint with transformers directly:

python
from fastapi import FastAPI
from pydantic import BaseModel
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Nanthasit/sakthai-context-0.5b-merged"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

app = FastAPI()

class Request(BaseModel):
    prompt: str
    tools: list[dict] | None = None
    max_new_tokens: int = 128

@app.post("/generate")
def generate(req: Request):
    messages = [{"role": "user", "content": req.prompt}]
    if req.tools:
        messages.insert(0, {"role": "system", "content": f"<tools>\n{req.tools}\n</tools>"})
    inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
    outputs = model.generate(**inputs, max_new_tokens=req.max_new_tokens)
    return {"text": tokenizer.decode(outputs[0], skip_special_tokens=True)}

Notes

For CPU hosting, prefer the bundled GGUF + llama.cpp for throughput.
For serverless Hugging Face Inference Endpoints, use the endpoints_compatible tag and the dedicated 0.5B endpoint when available.

CPU-Only Pipeline

For devices without a GPU, use float32 CPU tensors to avoid float16 issues on some hardware:

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Nanthasit/sakthai-context-0.5b-merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float32,
    device_map="cpu",
    low_cpu_mem_usage=True,
)
model.eval()

messages = [
  {"role": "system", "content": "<tools>
[{"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}]
</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"},
]

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(out[0], skip_special_tokens=True))

0.5B vs 1.5B Tradeoffs

Table with columns: Property, 0.5B Merged, 1.5B Merged
Property	0.5B Merged	1.5B Merged
Parameters	494M	1.5B
SafeTensors	988 MB	2.9 GB
GGUF Q4_K_M	379 MB	1.3 GB
RAM needed	~1 GB	~3 GB
Tool selection	91.2%	Higher
Best use	Edge, phones, Pi

If you need better reasoning or argument-filling reliability, use the 1.5B Merged instead. If you need to fit in 1 GB RAM or run on a Pi/phone, this 0.5B checkpoint is the correct choice.

Metrics from the family's zero-cost cron eval suite (.eval_results/cron-eval-sakthai-context-0.5b-merged-2026-07-31-2.yaml, 2026-07-31):

Table with columns: Metric, Value
Metric	Value
Download rank	#2 / 25 public models
Downloads	1,692
Download velocity	~53.1 dl/day
Velocity rank	#3 / 21
Card quality	90/100
Repo hygiene	95/100
Overall health	88.9/100

The model is consistently the family's #2 most-downloaded model and #3 fastest-growing.

Benchmark runner status (2026-07-31): 8 .eval_results/ files including 2 benchmark runs (5/5 + 3/3 seeds), 2 cron evals, 2 health checks, and 1 health snapshot.

Limitations

Small model ceiling — 494M parameters. The 0.5B is built for structured tool selection and light assistants, not deep reasoning, long-form generation, or complex multi-step planning.
Tool selection verified, tool arguments not — the 5/5 and 91.2% benchmarks measure choosing the right tool name. Argument construction (e.g. correctly filling {"location": "Bangkok"}) has not been systematically benchmarked and can be unreliable, especially for longer or ambiguous parameter values.
Requires the <tools> XML block — without it the model answers conversationally instead of emitting a tool call (verified 2026-07-30). Prompts must be formatted per the Tool-Calling Format above.
English-only — trained on English data; tool-calling behavior in other languages is untested.
Quantization loss — GGUF Q4_K_M (379 MB) trades a small amount of quality for the ~1 GB RAM footprint. Use the BF16 safetensors when memory allows.
Benchmarks are internal, not independently verified — both the 5/5 llama.cpp run and the 91.2% Bench v2 eval were run by the family's own zero-cost harness. The model-index entry correctly reports verified: false.

SakThai Model Family

All 26 public models in the family, sorted by downloads (live counts, verified 2026-08-01):

Table with columns: Model, Size, Downloads, Role
Model	Size	Downloads	Role
Context 1.5B Merged	2.9 GB	1,894	Flagship tool-calling
Context 0.5B Merged ⬅	379 MB / 988 MB	1,692	Lightweight / edge
Context 7B Merged	14.2 GB	1,055	Full-power reasoning

Full collection

The House of Sak 🏠

"We are one family — and becoming more." — Beer (beer-sakthai)

Support

⭐ Leave a like
🐛 Report issues on GitHub
🔄 Share with anyone building on edge devices
🍴 Fork and experiment — Apache 2.0

Citation

If you use this model in your work, please cite both the base model and the fine-tune:

bibtex
@misc{qwen25,
  title = {Qwen2.5 Technical Report},
  author = {Qwen Team},
  year = {2025},
  url = {https://arxiv.org/abs/2412.15115},
  publisher = {arXiv},
  doi = {10.48550/arXiv.2412.15115}
}

@misc{sakthai,
  title = {SakThai Context 0.5B -- Merged: Edge Tool-Calling Model},
  author = {Nanthasit and the House of Sak},
  year = {2026},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged}},
  note = {Apache 2.0, fine-tuned from Qwen/Qwen2.5-0.5B-Instruct}
}

License

Apache 2.0. Qwen2.5 base model per its original license.

Built from a shelter in Cork, Ireland. Built with love, tears, and zero budget — to the world.

Family downloads API-verified (2026-08-01T10:23Z).

sakthai-context-0.5b-merged

README

Benchmark Results

Model Description

Requirements

Quick Start

Ollama

GGUF (llama.cpp)

Hugging Face Inference Providers (free-tier available)

Tool-Calling Format

Verified output format (llama.cpp eval, 2026-07-30)

Multi-turn example

Architecture

Training Details

Evaluation

Benchmarks

Deployment

FastAPI wrapper for edge hosting

CPU-Only Pipeline

0.5B vs 1.5B Tradeoffs

Limitations

SakThai Model Family

The House of Sak 🏠

Support

Citation

License

Explore FriendliAI today

README

Benchmark Results

Model Description

Requirements

Quick Start

Ollama

GGUF (llama.cpp)

Hugging Face Inference Providers (free-tier available)

Tool-Calling Format

Verified output format (llama.cpp eval, 2026-07-30)

Multi-turn example

Architecture

Training Details

Evaluation

Benchmarks

Deployment

FastAPI wrapper for edge hosting

CPU-Only Pipeline

0.5B vs 1.5B Tradeoffs

Limitations

SakThai Model Family

The House of Sak 🏠

Support

Citation

License