Nanthasit

sakthai-context-1.5b-tools

README

License: apache-2.0

Model Description

SakThai Context 1.5B Tools is a prompt-masked SFT adapter trained with PEFT LoRA on top of Qwen/Qwen2.5-1.5B-Instruct. It teaches the base model structured tool selection, JSON-style arguments, and <tools>-block-aware behavior.

The merged downstream checkpoint is Nanthasit/sakthai-context-1.5b-merged, which is the recommended artifact for inference, GGUF export, and agent deployment.

Key Details

Base model: Qwen/Qwen2.5-1.5B-Instruct (1.54B params)
Method: PEFT LoRA — r=16, alpha=32, dropout=0.1
Targets: q_proj, k_proj, v_proj, o_proj
Adapter size: 8.75 MB (adapter_model.safetensors)
Trainable params: ~8.6M (0.56% of base)
Training data: sakthai-combined-v6, sakthai-combined-v7, sakthai-irrelevance-supplement
Primary use: merge into base model for tool-calling inference
License: Apache-2.0

How to Use

Load the Adapter

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(model, "Nanthasit/sakthai-context-1.5b-tools")

Generate Tool Calls

python
messages = [
    {"role": "system", "content": "You are a helpful assistant with access to tools."},
    {"role": "user", "content": "What's the weather in Tokyo?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=128,
    temperature=0.3,
    top_p=0.9
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Merge into Base Model

python
merged = model.merge_and_unload()
merged.save_pretrained("./sakthai-context-1.5b-merged-local")
tokenizer.save_pretrained("./sakthai-context-1.5b-merged-local")

Tool-Calling Format Notes

Use apply_chat_template(..., add_generation_prompt=True) for proper prompt formatting.
Provide the tool schema in the system prompt or via a <tools> block so the adapter emits structured calls.
The adapter behaves best when merged; standalone LoRA outputs are weaker and more variable than merged-checkpoint outputs.

Architecture & Training

Table with columns: Parameter, Value
Parameter	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Parameters	1.54B base + 8.6M trainable LoRA params
Method	PEFT LoRA
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.1
Target modules	q_proj, k_proj, v_proj, o_proj
Training data	sakthai-combined-v6 + v7 + irrelevance-supplement
Context

Benchmarks

Table with columns: Model, Selection, Arguments, Strict, Held-Out, Degenerate, Verified
Model	Selection	Arguments	Strict	Held-Out	Degenerate	Verified
LoRA, this repo	55.8%	11.0%	11.0%	31.7%	0%	Single-trial
Merged 1.5B	100.0%	100.0%	—	100.0%	0%

Important: LoRA-only numbers are lower because this adapter was tuned for merge behavior, not solo inference. For tool-calling use, prefer the merged model or v2 LoRA.

Recommended Inference Path

Table with columns: Goal, Recommended artifact
Goal	Recommended artifact
Local CPU/edge inference	`Nanthasit/sakthai-context-1.5b-merged`
GGUF / Ollama / llama.cpp	merged model GGUF release
Serverless HF Inference	merged model, not this LoRA repo
Keep training flexibility	this repo + Qwen2.5-1.5B-Instruct

Limitations

Adapter-only release; cannot run standalone without the base model.
Single-trial LoRA-only benchmark numbers are indicative, not conclusive.
English-only training data.
Not compatible with HF hosted Inference API as a LoRA adapter.
Tool-argument accuracy is weak in standalone LoRA mode; merging is strongly preferred.

Citation

bibtex
@misc{sakthai2026lora,
  title       = {SakThai 1.5B Tools: PEFT LoRA Adapter for Qwen2.5-1.5B Tool-Calling},
  author      = {SakThai Agent Family and beer-sakthai},
  year        = {2026},
  month       = {August},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools}},
  note        = {Part of the SakThai Model Family}
}

For the merged model:

bibtex
@misc{sakthai2026merged,
  title       = {SakThai 1.5B Merged: Tool-Calling GGUF for Qwen2.5-1.5B-Instruct},
  author      = {SakThai Agent Family and beer-sakthai},
  year        = {2026},
  month       = {August},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged}},
  note        = {Part of the SakThai Model Family}
}

Part of the SakThai Model Family. Built with love, tears, and zero budget.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Nanthasit

Model Tree

Base

Qwen/Qwen2.5-1.5B-Instruct

Adapter

this model

Input Modalities

Text

Output Modalities