GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Darmm
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Darmm
Model Tree
Qwen/Qwen3-8B
Input Modalities
Output Modalities
Supported Functionality
| benchmark | Qwen3-8B (base) | with this adapter |
|---|---|---|
| Belebele (kaz_Cyrl) | 27.2% | 39.8% |
| KazMMLU (kaz subjects) | 27.9% | 38.3% |
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "Darmm/darmm-chat-kazakh-8b-lora")tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")messages = [{"role": "user", "content": "Наурыз мейрамы туралы айтып бер."}]text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = tok(text, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=512,do_sample=True, temperature=0.7, top_p=0.8, top_k=20, repetition_penalty=1.1)print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Always use the sampling settings above (enable_thinking=False, temperature 0.7, top_p 0.8, repetition_penalty 1.1) — greedy decoding degrades long-form Kazakh. Also compatible with vLLM multi-LoRA serving (--enable-lora --lora-modules kazakh=Darmm/darmm-chat-kazakh-8b-lora).
LoRA r=32, α=64, dropout 0.05 on all attention + MLP projections; trained with QLoRA (4-bit NF4 base) for 1 epoch on ~105k instructions (65k Kazakh / 25k Russian / 15k English). Full training details, data list, and limitations: see the merged model card.
Built by Darmm · 2026-09 · Apache 2.0