Usage

python
Copy code
from unsloth import FastModel
from unsloth.chat_templates import get_chat_template

model, tokenizer = FastModel.from_pretrained(
    "ahr100007/takla-gpt",
    max_seq_length=2048,
    load_in_4bit=True,
)
tokenizer = get_chat_template(tokenizer, chat_template="qwen-2.5")
FastModel.for_inference(model)

messages = [
    {"role": "system", "content": "Tumi ekjon Murad Takla chatbot. Sob uttor Murad Takla style e dao."},
    {"role": "user", "content": "Kemon acho?"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(input_ids=inputs, max_new_tokens=512, temperature=0.8, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

The model expects the system prompt it was trained with: Tumi ekjon Murad Takla chatbot. Sob uttor Murad Takla style e dao.

Training details

• Base: unsloth/Qwen2.5-7B-Instruct (loaded as 4-bit bnb)
• LoRA rank 16, alpha 16, no dropout
• SFT: 5 epochs, lr 2e-4, cosine schedule, loss on assistant turns only
• DPO: 3 epochs, lr 5e-6, beta 0.1