🇹🇲 TurkmenAI stack
Evaluation
Base-vs-finetuned comparison is done by toggling the LoRA adapter on the
same model with the same prompts and greedy decoding — a controlled,
reproducible A/B.
Table with columns: Metric, Qwen/Qwen2.5-3B-Instruct (base), TurkmenChat-3B, Delta| Metric | Qwen/Qwen2.5-3B-Instruct (base) | TurkmenChat-3B | Delta |
|---|
| FLORES-200 en->tk chrF++ (200 sent.) | 14.40 | 15.20 | +0.8 |
| Held-out perplexity (2% split) | - | n/a | - |

Full numbers: benchmark_results.json. Sample
translations: flores_translations_sample.json.
Sample generations
Salam, näme habar?
Salam aleykum! Myn name is John and I am doing great. How about you?
Aşgabat hakynda gysgaça gürrüň ber.
Aşgabat (previously known as Stanyazovka) is the capital and largest city of Turkmenchat, a republic in Central Asia. It is located near the borders with Uzbekistan and Kazakhstan. The city has a population of around 600,000 people.
1-den 10-a çenli sanap ber.
1,2,3,4,5,6,7,8,9,10
Plow nädip bişirmeli?
Yes, a plow is used to till the soil and prepare it for planting crops. It has two parts: the blade that cuts into the ground and the coulter which cuts into the soil at an angle. The plow is pulled by a tractor or other machinery and can be adjusted in order to achieve the desired depth of cut.
Usage
from transformers import pipeline
chat = pipeline("text-generation", model="Slavik2009/TurkmenChat-3B")
messages = [{"role": "user", "content": "Salam! Aşgabat hakynda gürrüň ber."}]
print(chat(messages, max_new_tokens=200)[0]["generated_text"][-1]["content"])
CPU inference (Ollama / llama.cpp)
ollama run hf.co/Slavik2009/TurkmenChat-3B-GGUF:Q4_K_M
Training details
- Data: mamed0v/alpaca-turkmen
(52k instructions), OPUS-100 en-tk
translation pairs, native-speaker dialog corpus (upweighted)
- Method: QLoRA 4-bit (Unsloth), r=32/alpha=64, all attention+MLP projections,
cosine LR schedule, sequence packing, dedup + length filtering
- Benchmark: FLORES-200 (Meta's standard benchmark for low-resource MT);
chrF++ — the accepted metric for morphologically rich languages
- Hardware: Tesla T4 (Kaggle free tier), incremental sessions with
adapter checkpointing to the Hub
Intended use & limitations
Built for casual Turkmen conversation, Q&A and en->tk translation. Not tuned
for factual reliability, safety-critical use, code, or long documents.
It can hallucinate — verify important facts. Latin-script Turkmen only.
Citation
@misc{turkmenchat3b,
title = {TurkmenChat-3B: an open conversational LLM for Turkmen},
author = {Slavik2009},
year = {2026},
url = {https://huggingface.co/Slavik2009/TurkmenChat-3B}
}