GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ThaiLLM
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ThaiLLM
Model Tree
Input Modalities
Output Modalities
Supported Functionality
The training corpus consists of the following datasets:
| Dataset | Tokens |
|---|---|
| Fineweb2-ENG | 24,000,000,000 |
| Fineweb2-TH | 31,525,674,209 |
| CuratedData | 8,054,246,789 |
| Category | Token Count |
|---|---|
| Business & Finance | 736,071,807 |
| News | 1,700,662,378 |
| Education | 576,489,778 |
| Social | 211,000,000 |
| Government | 40,492,117 |
| Medical | 42,987,587 |
| Conversation | 80,919,390 |
| Code | 620,218 |
| Research Articles | 4,185,649,758 |
| Law | 467,994,847 |
| Travel | 6,948,290 |
| Others | 4,410,619 |
*Token counts calculated using Qwen3 Tokenizer
The code of Qwen3 has been integrated into the latest Hugging Face transformers library. We strongly recommend using the latest version of transformers.
With transformers<4.51.0, you will encounter the following error:
text
KeyError: 'qwen3'Important: This is a base model and requires instruction fine-tuning before use to ensure optimal performance for your specific tasks and requirements.
We recommend using LLaMA-Factory for instruction fine-tuning. This framework provides an easy-to-use interface for training language models with various optimization techniques.
bash
# Clone the repositorygit clone https://github.com/hiyouga/LLaMA-Factory.gitcd LLaMA-Factory # Install dependenciespip install -e . # Example training command for LoRAllamafactory-cli train \ --model_name_or_path ThaiLLM/ThaiLLM-8B \ --stage sft \ --do_train \ --finetuning_type lora \ --dataset your_dataset \ --template qwen3 \ --cutoff_len 8192 \ --learning_rate 5e-05 \ --num_train_epochs 3.0 \ --per_device_train_batch_size 2 \ --gradient_accumulation_steps 8 \ --lr_scheduler_type cosine \ --max_grad_norm 1.0 \ --logging_steps 5 \ --save_steps 100 \ --warmup_steps 0 \ --output_dir saves/ThaiLLM-8B-lora \ --bf16Below are code snippets to get quickly started with running the model. First, install the necessary libraries.
bash
pip install -U transformers torch acceleratepython
from transformers import AutoTokenizer, AutoModelForCausalLM, import torch model_id = "ThaiLLM/ThaiLLM-8B" # Load model and tokenizertokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", torch_dtype=torch.bfloat16) # Example promptprompt = "น้ำบริสุทธิ์มีค่า pH เท่าใด"inputs = tokenizer(prompt, return_tensors="pt") # Generate responsewith torch.inference_mode(): generate_ids = model.generate( inputs.input_ids, max_new_tokens=500, repetition_penalty=1.2, num_beams=1, do_sample=True, top_k=40, top_p=0.75, temperature=0.4, pad_token_id=tokenizer.eos_token_id, ) response = tokenizer.batch_decode( generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True)[0] print(response)We evaluated ThaiLLM-8B against Qwen3-8B-Base using multiple-choice question datasets in both Thai and English.
Each benchmark measures the probability of selecting the correct choice based on the model’s next-token prediction.
| Task | Qwen3-8B-Base | ThaiLLM-8B | Δ |
|---|---|---|---|
| MMLU (ENG, 5-shot) | 0.7691 | 0.7565 | -0.0126 |
| MMLU (TH) | 0.6259 | 0.6459 | +0.0200 |
| ThaiExam Avg. (ONET, IC, TGAT, TPAT-1, A-Level) | 0.31396 | 0.48292 | +0.16896 |
| ├── ONET | 0.4074 | 0.5864 | +0.1790 |
| ├── IC | 0.5157 | 0.7052 | +0.1895 |
| ├── TGAT | 0.3384 | 0.6307 | +0.2923 |
| ├── TPAT-1 | 0.1379 | 0.3965 | +0.2586 |
| └── A-Level | 0.1653 | 0.5275 | +0.3622 |
| M3Exam | 0.5802 | 0.6369 | +0.0567 |
| M6Exam Avg. | 0.54844 | 0.55792 | +0.00948 |
| ├── Thai | 0.4833 | 0.5023 | +0.0190 |
| ├── Math | 0.4090 | 0.2727 | -0.1363 |
| ├── Social | 0.5844 | 0.7088 | +0.1244 |
| ├── Science | 0.4603 | 0.5238 | +0.0635 |
| └── English | 0.7552 | 0.7864 | +0.0312 |
| XNLI-Thai | 0.7529 | 0.6667 | -0.0862 |
| XCOPA-Thai | 0.8220 | 0.8340 | +0.0120 |
| Belebele-Thai | 0.3880 | 0.8447 | +0.4567 |
| Model | Average Score |
|---|---|
| Qwen3-8B-Base | 0.5987 |
| ThaiLLM-8B | 0.6891 |
Highlights:
- ThaiLLM-8B shows large improvements in ThaiExam, Belebele-Thai, and MMLU-TH.
- Gains are especially strong in A-Level (+0.36) and TGAT (+0.29).
- Some slight regressions are seen in MMLU-ENG and Math in M6Exam.
bibtex
@misc{qwen3technicalreport, title={Qwen3 Technical Report}, author={Qwen Team}, year={2025}, eprint={2505.09388}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2505.09388}, }
# Clone the repositorygit clone https://github.com/hiyouga/LLaMA-Factory.gitcd LLaMA-Factory # Install dependenciespip install -e . # Example training command for LoRAllamafactory-cli train \ --model_name_or_path ThaiLLM/ThaiLLM-8B \ --stage sft \ --do_train \ --finetuning_type lora \ --dataset your_dataset \ --template qwen3 \ --cutoff_len 8192 \ --learning_rate 5e-05 \ --num_train_epochs 3.0 \ --per_device_train_batch_size 2 \ --gradient_accumulation_steps 8 \ --lr_scheduler_type cosine \ --max_grad_norm 1.0 \ --logging_steps 5 \ --save_steps 100 \ --warmup_steps 0 \ --output_dir saves/ThaiLLM-8B-lora \ --bf16from transformers import AutoTokenizer, AutoModelForCausalLM, import torch model_id = "ThaiLLM/ThaiLLM-8B" # Load model and tokenizertokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", torch_dtype=torch.bfloat16) # Example promptprompt = "น้ำบริสุทธิ์มีค่า pH เท่าใด"inputs = tokenizer(prompt, return_tensors="pt") # Generate responsewith torch.inference_mode(): generate_ids = model.generate( inputs.input_ids, max_new_tokens=500, repetition_penalty=1.2, num_beams=1, do_sample=True, top_k=40, top_p=0.75, temperature=0.4, pad_token_id=tokenizer.eos_token_id, ) response = tokenizer.batch_decode( generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True)[0] print(response)@misc{qwen3technicalreport, title={Qwen3 Technical Report}, author={Qwen Team}, year={2025}, eprint={2505.09388}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2505.09388}, }