Hyperparameters
{
"lr": 1e-05,
"epochs": 1,
"r": 16,
"qlora": true,
"mix": "30:70",
"bs": 4,
"ga": 2,
"max_len": 1536
}
Evaluation (top-3 benchmarks)
Table with columns: Benchmark, Score, Target, Beat?| Benchmark | Score | Target | Beat? |
|---|
| thaiexam | 35.93 | 75.0 | ❌ |
| math500 | 54.80 | 82.0 | ❌ |
| aime24 | 23.33 | 25.0 | ❌ |
Recipe summary
- LoRA rank: 16, alpha: —, dropout: —
- Learning rate: 1e-05, epochs: 1
- QLoRA 4-bit (nf4): True
- Mix Thai:EN: 30:70
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "deepseek-ai/DeepSeek-R1-Distill-Qwen-14B"
adapter = "Jnx03/kanitakorn-r1d-qwen14b-stage1-v1b-20260521"
tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, adapter)
Campaign log
See VERSION_LOG.md for context on this checkpoint vs siblings.
License: MIT (inherited from deepseek-ai/DeepSeek-R1-Distill-Qwen-14B).