Model Description
Table with columns: Attribute, Value| Attribute | Value |
|---|
| Base model | Qwen/Qwen2.5-1.5B-Instruct |
| Parameters | 1.5B |
| Adapter | LoRA (r=16, alpha=32, dropout=0.05) |
| LoRA target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| LoRA variant | rslora |
| Context length | 32,768 tokens |
| Method | SFT via TRL |
| License | Apache 2.0 |
| Output | Text generation |
Usage
Apply LoRA adapter
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_id = "Qwen/Qwen2.5-1.5B-Instruct"
lora_id = "Nanthasit/sakthai-plus-1.5b-lora"
base_model = AutoModelForCausalLM.from_pretrained(base_id)
tokenizer = AutoTokenizer.from_pretrained(base_id)
model = PeftModel.from_pretrained(base_model, lora_id)
model = model.merge_and_unload()
inputs = tokenizer("You are a helpful assistant.", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Inference via merged artifact
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Nanthasit/sakthai-plus-1.5b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
prompt = "Plan a 3-day trip to Lisbon with a €200 budget."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Inference via Hugging Face Inference API
curl -X POST https://api-inference.huggingface.co/models/Nanthasit/sakthai-plus-1.5b \
-H "Authorization: Bearer ***" \
-H "Content-Type: application/json" \
-d '{"inputs": "Your prompt here"}'
Benchmarks
Table with columns: Benchmark, Value, Status| Benchmark | Value | Status |
|---|
| Tool Calling — send_email (SakThai Bench v2) | 1.0 | verified |
| General Assistant — text generation | pending | multi-trial eval ongoing |
Evidence files: .eval_results/cron-eval-2026-07-31-2.yaml.
See the leaderboard Space for live updates as scores are verified.
Evaluation
- Conversation: Multi-turn dialogue, follow-up handling, context recall
- Reasoning: Step-by-step problem-solving, creative answers
- Instruction Following: SFT-tuned for clear, concise responses
- Inference Speed: ~50-100 tokens/sec on CPU; ~500+ tokens/sec on GPU
- Tool Use: Inherited merged sibling shows full-tool-call accuracy on send_email benchmark
Training Data: SakThai Combined v7 (2,003 examples — tool-use, Q&A, multi-turn conversation, reasoning tasks)
Training Details
Table with columns: Item, Value| Item | Value |
|---|
| Base | Qwen/Qwen2.5-1.5B-Instruct |
| Dataset | Nanthasit/sakthai-combined-v7 |
| Framework | TRL + PEFT 0.19.1 |
| Adapter type | LoRA + rslora |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| Dropout | 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Task type |
Reproduce Evaluation
python scripts/eval_tool_calling.py \
--model_id Nanthasit/sakthai-plus-1.5b-lora \
--base_model_id Qwen/Qwen2.5-1.5B-Instruct \
--dataset Nanthasit/sakthai-bench-v3 \
--task_type tool_calling \
--trials 3
Reproduce Training / Merge
python scripts/train_lora.py \
--base_model Qwen/Qwen2.5-1.5B-Instruct \
--dataset Nanthasit/sakthai-combined-v7 \
--output_dir lora-out
python scripts/merge_lora.py \
--base_model Qwen/Qwen2.5-1.5B-Instruct \
--adapter_dir lora-out \
--output_dir merged
Limitations
- LoRA has lower capacity than full fine-tuning — complex multi-step reasoning may benefit from a merged+quantized version
- Best results when base model is loaded with appropriate prompt template
- No instruction-only mode — always use base model's chat template
- Tool-calling benchmark here is reproduced on the merged checkpoint, not adapter-only inference
- Generated content can be inaccurate; verify factual claims independently
Citation
@misc{sakthai-plus-1.5b-lora,
title = {SakThai Plus 1.5B LoRA},
author = {Beer Nanthasit and SakThai Agents},
year = {2026},
url = {https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora
}
Family & Links
License
Apache License 2.0. Derived from Qwen/Qwen2.5-1.5B-Instruct (Apache 2.0).
SakThai Agents always tell you: this model exists because someone chose to build with no money and infinite heart. Use it well. 💜