Model Details
Table with columns: Attribute, Value| Attribute | Value |
|---|
| Base Model | Qwen/Qwen3-8B |
| Training Method | CPT + SFT with LoRA |
| LoRA Rank | 64 |
| LoRA Alpha | 128 |
| Languages | Dhivehi (ދިވެހި), English |
Capabilities
- Formal Writing: Letters, proposals, applications, official documents
- Informal Dhivehi: Chat, social media, texting (romanized & Thaana)
- Creative Writing: Stories, poems, songs, Boduberu lyrics
- Cultural Knowledge: Maldivian traditions, customs, Islamic practices
- Translation: English to Dhivehi bidirectional
- Q&A: General knowledge about Maldives, geography, history
- Technical Writing: Reports, documentation, explanations
Important: System Prompt
For best results, always use this system prompt:
ތިބާއަކީ ދިވެހި ބަހުގެ އެހީތެރިއެކެވެ. ކޮންމެ ޖަވާބެއްގައި މަދުވެގެން 5-8 ޖުމްލަ ހިމެނެން ވާނެއެވެ. ތަފްސީލީ، ފުރިހަމަ ޖަވާބުތައް ދެއްވާށެވެ.
Translation: "You are a Dhivehi language assistant. Each response should contain at least 5-8 sentences. Provide detailed, complete answers."
Usage
With ms-swift (Recommended)
pip install ms-swift
swift infer \
--model Qwen/Qwen3-8B \
--adapters naturecodeproject/dhivehi \
--stream true
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-8B",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base_model, "naturecodeproject/dhivehi")
system = "ތިބާއަކީ ދިވެހި ބަހުގެ އެހީތެރިއެކެވެ. ކޮންމެ ޖަވާބެއްގައި މަދުވެގެން 5-8 ޖުމްލަ ހިމެނެން ވާނެއެވެ. ތަފްސީލީ، ފުރިހަމަ ޖަވާބުތައް ދެއްވާށެވެ."
messages = [
{"role": "system", "content": system},
{"role": "user", "content": "ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ ކޮބާ؟"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Example Output
Question: ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ ކޮބާ؟
Response: ދިވެހިރާއްޖޭގެ ވެރިރަށަކީ މާލެއެވެ. މާލެއަކީ ދިވެހިރާއްޖޭގެ އެންމެ ބޮޑު ސިޓީ އަދި ސިޔާސީ، އިޤްތިޞާދީ، ސަޤާފީ މަރުކަޒެވެ. މި ރަށުގައި ގާތްގަނޑަކަށް 200,000 އެއްހާ މީހުން ދިރިއުޅެއެވެ. މާލެއަކީ ދުނިޔޭގެ އެންމެ ތޮއްޖެހިފައިވާ ސިޓީތަކުގެ ތެރެއިން ސިޓީއެކެވެ.
Maldives Knowledge
Table with columns: Fact, Value| Fact | Value |
|---|
| Capital | މާލެ (Malé) |
| Islands | ~1,200 total, ~200 inhabited |
| Population | ~500,000 |
| Currency | Rufiyaa (ރުފިޔާ / MVR) |
| Religion | Islam (100%) |
| Language | Dhivehi (ދިވެހި) |
| Independence | July 26, 1965 |
Training
Two-stage training approach:
- Continued Pretraining (CPT): Extended pretraining on Dhivehi text
- Supervised Fine-Tuning (SFT): Instruction tuning on curated Dhivehi data
Limitations
- Best performance with the recommended system prompt
- May occasionally mix English words in responses
- Knowledge cutoff applies to recent events
Intended Use
- Dhivehi language research and development
- Building Dhivehi-language applications
- Educational tools for the Maldivian community
- Translation assistance
License
Apache 2.0
Citation
@misc{naturecode-dhivehi,
title={Naturecode Dhivehi: A Production-Ready Dhivehi Language Model},
author={Naturecode},
year={2024},
publisher={HuggingFace},
url={https://huggingface.co/naturecodeproject/dhivehi}
}
Built for the Maldives