Nanthasit

sakthai-context-1.5b-merged-v2

README

License: apache-2.0

Model Description

SakThai Context 1.5B Merged V2 is a merged full-weight checkpoint of Qwen2.5-1.5B-Instruct, fine-tuned for structured tool-calling and function-calling. It fills the mid-weight slot between the edge-focused 0.5B and the high-capability 7B. Use it when you need stronger argument filling and multi-turn tool behavior on a laptop/desktop with 4–8 GB RAM.

What makes it special:

🧠 1.5B parameters — stronger reasoning than 0.5B, smaller than 7B.
🗳️ Trained for structured <tool> / <tool_call> output.
📦 GGUF Q4_K_M + BF16 safetensors included.
✅ Benchmarks available: Bench v2 selection 34.9%, arguments 44.2%, strict 34.2%; internal smoke test 100% multi-tool correctness / valid JSON.
🔁 Rebuilt merged weights, ready for CPU inference with llama.cpp or transformers.

Requirements

Use these tested versions to avoid inference issues on CPU/edge hardware:

text
torch>=2.2
transformers>=4.45
sentencepiece>=0.2
protobuf>=3.20
accelerate>=0.27

For GGUF inference:

text
llama-cpp-python>=0.2.80

Quick Start

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Nanthasit/sakthai-context-1.5b-merged-v2"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
  {"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Ollama

The model is not published to the Ollama library, so import the local GGUF instead:

bash
ollama create sakthai:1.5b-v2 -f Modelfile

Modelfile:

markdown
FROM ./sakthai-1.5b-q4_k_m.gguf

GGUF (llama.cpp)

bash
# Option A — huggingface-cli
huggingface-cli download Nanthasit/sakthai-context-1.5b-merged-v2 --include "*.gguf" --local-dir ./
# Option B — direct wget
wget https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2/resolve/main/sakthai-1.5b-q4_k_m.gguf

llama-cli -m sakthai-1.5b-q4_k_m.gguf \
  --prompt "<|system|>You are SakThai-Agent.<|user|>What's the weather in Bangkok?<|assistant|>" -n 256

Hugging Face Inference Providers (free-tier available)

Use the hosted Inference API with huggingface_hub.InferenceClient:

python
from huggingface_hub import InferenceClient

client = InferenceClient(model="Nanthasit/sakthai-context-1.5b-merged-v2")
messages = [
  {"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"},
]
response = client.chat_completion(messages=messages, max_tokens=256)
print(response.choices[0].message.content)

Tool-Calling Format

The model is fine-tuned for tool calling and expects a <tools> XML block in the system prompt — the block is required: without it, the model may answer conversationally instead of emitting a tool call.

Verified output format

markdown
System: <tools>
[
  {"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}
]
</tools>
User: What's the weather in Bangkok?
Assistant: <tool>get_weather</tool>

For calls with arguments, the bundled chat_template.jinja renders the standard Qwen <tool_call> JSON form:

markdown
<tool_call>
{"name": "get_weather", "arguments": {"location": "Bangkok"}}

Architecture

Table with columns: Property, Value
Property	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Parameters	1.5B (1,500,000,000)
Hidden size	1,536
Layers	28
Attention heads	12 (grouped-query, 2 KV heads)
Intermediate size	8,960
Context window	32,768 tokens
Vocab size	151,936

Training Details

Table with columns: Detail, Value
Detail	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Method	SFT → merged to full weights
Training data	sakthai-combined-v6/v7 + irrelevance-supplement
Hardware	Free T4 GPU (Kaggle / Colab)

Evaluation

Table with columns: Setting, Selection, Arguments, Strict, Valid JSON, Multi-tool
Setting	Selection	Arguments	Strict	Valid JSON	Multi-tool
SakThai Bench v2	34.9%	44.2%	34.2%	—	—
Internal smoke test	—	—	—	100%	100%

Notes:

Benchmarks are internal and should not be treated as independently verified.
model-index uses verified: true only because the corresponding .eval_results/ files are present in the repo.

Deployment

CPU-only

python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Nanthasit/sakthai-context-1.5b-merged-v2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float32,
    device_map="cpu",
    low_cpu_mem_usage=True,
)
model.eval()
messages = [
  {"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Edge / llama.cpp

bash
huggingface-cli download Nanthasit/sakthai-context-1.5b-merged-v2 --include "*.gguf" --local-dir ./
llama-cli -m sakthai-1.5b-q4_k_m.gguf --prompt "<|system|>You are SakThai-Agent.<|user|>What's the weather in Bangkok?<|assistant|>" -n 256

1.5B vs 7B Tradeoffs

Table with columns: Property, 1.5B Merged V2, 7B Merged
Property	1.5B Merged V2	7B Merged
Parameters	1.5B	7B
SafeTensors	2.9 GB	14.2 GB
GGUF Q4_K_M	~1.3 GB	~4.6 GB
RAM needed	~3 GB	~8 GB
Tool selection	34.9%	Higher
Best use	Laptop/desktop, balanced CPU inference

If you need stronger tool argument accuracy and more headroom, use the 7B Merged. If you need the smallest footprint, use the 0.5B Merged.

Limitations

Smaller than 7B — still has reasoning limits compared to larger checkpoints.
Benchmark numbers are internal — Bench v2 and smoke-test results are not independently verified.
Requires <tools> XML block — without it, the model defaults to conversation mode.
English-only behavior — untested in other languages for tool calls.
Argument accuracy is lower than selection — Arguments Accuracy 44.2% indicates the model often picks the right tool but may misconstruct parameters.

SakThai Model Family

All 26 public models in the family, sorted by downloads (live counts, verified 2026-08-01):

Table with columns: Model, Size, Downloads, Role
Model	Size	Downloads	Role
Context 1.5B Merged	3.8 GB	1,894	Flagship tool-calling
Context 0.5B Merged	1.3 GB	1,730	Lightweight / edge
Context 7B Merged	14.2 GB	1,055

Full collection

The House of Sak 🏠

This model is part of the House of Sak — an open-source AI ecosystem built from a shelter in Cork, Ireland, with $0 budget and no paid GPUs.

"We are one family — and becoming more." — Beer (beer-sakthai)

Support

⭐ Leave a like
🐛 Report issues on GitHub
🔄 Share with anyone building accessible AI on CPU/edge
🍴 Fork and experiment — Apache 2.0

Citation

bibtex
@misc{sakthai-context-1.5b-merged-v2,
  title = {SakThai Context 1.5B -- Merged V2: Mid-Weight Tool-Calling Model},
  author = {Nanthasit and the House of Sak},
  year = {2026},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2}},
  note = {Apache 2.0, fine-tuned from Qwen/Qwen2.5-1.5B-Instruct}
}

License

Apache 2.0. Qwen2.5 base model per its original license.

Built from a shelter in Cork, Ireland. Built with love, tears, and zero budget — to the world.

Family downloads API-verified (2026-08-01T10:23Z).

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.