Muhammad-Zubair796

Khatta-ka

README

License: apache-2.0

👤 Developer Profile

Name: Muhammad Zubair
Profession: Software Engineer
Portfolio: mzubair.online
Project Case Study: Khatta-ka-LLM by Muhammad Zubair

🌟 Model Description

Standard Pashto AI models often fail to understand regional dialects. Khatta-ka-LLM was developed by Muhammad Zubair to bridge this gap. It understands the unique vocabulary, spelling, and grammar of the Khattak Pashto dialect (e.g., using "شوشته" and "ایس نه کاون").

Built on top of junaid008/qehwa-pashto-llm (Qwen2.5-7B), Khatta-ka AI has been fine-tuned using LoRA via Unsloth to adapt to the Khattak dialect. It understands regional Khattak vocabulary, grammar shifts (such as using "یں" instead of "یم" for the first-person copula, and "کاوں" instead of "کوم"), and phonetic variations.

⚠️ Note: This model is a LoRA adapter. It is designed to take instructions in English or standard Pashto and reply in authentic Khattak Pashto.

✨ Capabilities

✅ Answers questions in Khattak Pashto ✅ Responds to English instructions in Khattak Pashto ✅ Responds to Standard Pashto instructions in Khattak dialect ✅ Understands regional vocabulary specific to Karak, Kohat, and Nowshera ✅ Trained 2x faster using Unsloth by Muhammad Zubair

📘 The Official Khattak Dialect Grammar & Rulebook

Here is the complete summary of the rules this AI has been trained on to understand the Khattak dialect:

1. Pronouns & Possession

My / Mine: Uses مو والا (Mo wala) instead of standard Zama.
To me: Uses موته (Mota) instead of standard Mala.
To you: Uses توته (Tota) instead of standard Tata.
From me: Uses رونه (Rona) instead of standard Rana.
We: Uses موخ (Moxh) instead of standard Mung.

2. The "Noon Ghunna" (ں) Rule (Crucial!)

Verbs (First Person): When saying "I do something", the verb ends in a nasal Noon Ghunna (ں) instead of the standard "M" (م).
- I do: کاوں (Kaon)
- I wash: وینزاں (Wenzan)
- I write: لېکاں (Lekan)
- I will come: روساں (Rosan)
The "I am" Copula: Standard Pashto uses یم (Yam). Khattak uses یں (Yay + Noon Ghunna). Example: Za khwash یں (I am happy).
Nouns: Nouns do not use the Noon Ghunna. They keep the hard "N" (ن). Example: Asmon (اسمون), Baron (بارون), Pashtun (پښتون).

3. Tenses & Verbs

Future Tense: Uses بو (Bo) instead of standard به (Ba). Example: Za bo sabo char kaon (I will do work tomorrow).
"To Say/Tell": Uses یای (Yai) for "he says" and ویاوں (Wayaon) for "I will tell".
"To Read": Uses یای (Yai) instead of standard Lwali. Example: Agha kitob yai (He is reading a book).

4. Phonetics & Vowel Shifts

The "O" Shift: Many standard "A" sounds shift to a deep "O" sound.
- Asmaan ➡️ Asmon (اسمون)
- Baraan ➡️ Baron (بارون)
- Bazaar ➡️ Bazor (بازور)
- Maakham ➡️ Mashom (ماښوم)
- Yaada ➡️ Yoda (یوده)
Masculine Singular Nouns: Words that end in "i" in standard Pashto often end in "a" in Khattak.
- Gadi (Car) ➡️ Gada (ګاډه)
- Kanri (Stone) ➡️ Konrha (کاڼه)
- Halak/Kakai (Boy) ➡️ Orhka (اوړکه)

5. Unique Khattak Vocabulary

Family: Grandmother = انیو (Anyo)
People: Boy = اوړکه (Orhka), Girl = اوړکې (Orhkay)
Animals: Snake = منګور (Mangor)
Nature: Day = ورېز (Wrez)
Actions: Stand up = چګ شه (Chag sha), Flying = نغېږي (Neghezxhi), Crying = ژوړې (Zorhay)
Adjectives: Big = استر (Astr), Bad/Dirty = کسو (Kso), Tired (Singular) = ستړه (Starha)
Items: Tea leaves = پتې (Pathay)

💻 Installation

markdown
pip install transformers accelerate torch peft

For faster inference (Recommended):

markdown
pip install unsloth

🚀 How to Use

This model is an adapter. To use it, load it alongside the base model.

✅ Method 1 — Unsloth (Recommended - 2x Faster)

Best for: Speed-optimized usage, Colab, A100/H100/T4

markdown
from unsloth import FastLanguageModel
import torch

model_name = "Muhammad-Zubair796/Khatta-ka"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name     = model_name,
    max_seq_length = 2048,
    dtype          = None,
    load_in_4bit   = True,
)
FastLanguageModel.for_inference(model)

alpaca_prompt = """Below is an instruction. Write a detailed response in Pashto.

### Instruction:
{}

### Response:
"""

inputs = tokenizer(
    [
        alpaca_prompt.format("Where did the boy go?", "")
    ], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)
response = tokenizer.batch_decode(outputs, skip_special_tokens = True)[0]

khattak_output = response.split("### Response:\n")[-1].strip()
print(khattak_output)

✅ Method 2 — Standard Transformers & PEFT

Best for: Standard production environments without Unsloth

markdown
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_name = "junaid008/qehwa-pashto-llm"
adapter_name = "Muhammad-Zubair796/Khatta-ka"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype = torch.bfloat16,
    device_map  = "auto",
)

# Load the Khattak Adapter
model = PeftModel.from_pretrained(base_model, adapter_name)

alpaca_prompt = """Below is an instruction. Write a detailed response in Pashto.

### Instruction:
{}

### Response:
"""

def generate(prompt):
    inputs = tokenizer(
        alpaca_prompt.format(prompt, ""),
        return_tensors = "pt",
    ).to("cuda")

    outputs = model.generate(
        **inputs,
        max_new_tokens     = 64,
        temperature        = 0.7,
        do_sample          = True,
        pad_token_id       = tokenizer.eos_token_id,
    )

    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("### Response:\n")[-1].strip()

print(generate("Where did the boy go?"))

⚙️ Hardware Requirements

Table with columns: Method, VRAM, Speed
Method	VRAM	Speed
bfloat16 full	16GB+	✅ Fast
4-bit quantized	8GB+	✅ Good
Unsloth (4-bit)	8GB+	🔥 2x Faster

📊 Training Details

This model was trained using Unsloth for optimized, memory-efficient fine-tuning by Muhammad Zubair.

Table with columns: Parameter, Value
Parameter	Value
Base model	`junaid008/qehwa-pashto-llm`
Training Framework	Unsloth + TRL (SFTTrainer)
Epochs	4
Sequence length	2,048 tokens
LoRA Rank (r)	16
LoRA Alpha	16
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

🗣️ Example Outputs

English input:

Prompt: I am doing nothing, just sitting. Output: زه ایس نه کاوں، بس نوست یں

English input:

Prompt: Where did the boy go? Output: اوړکه چےتا لوړو؟

English input:

Prompt: The weather is very beautiful today, it is raining. Output: نن موسم ډېر شوشته دا، بارون وریږي

📜 License

Apache 2.0 — free to use, modify, and distribute with attribution.

🤝 Citation

markdown
@misc{khattaka-pashto-llm,
  author    = {Muhammad Zubair},
  title     = {Khatta-ka AI: Khattak Pashto LLM},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Muhammad-Zubair796/Khatta-ka}
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Muhammad-Zubair796

Model Tree

Base

junaid008/qehwa-pashto-llm

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

👤 Developer Profile

Name: Muhammad Zubair
Profession: Software Engineer
Portfolio: mzubair.online
Project Case Study: Khatta-ka-LLM by Muhammad Zubair

🌟 Model Description

⚠️ Note: This model is a LoRA adapter. It is designed to take instructions in English or standard Pashto and reply in authentic Khattak Pashto.

✨ Capabilities

📘 The Official Khattak Dialect Grammar & Rulebook

Here is the complete summary of the rules this AI has been trained on to understand the Khattak dialect:

1. Pronouns & Possession

My / Mine: Uses مو والا (Mo wala) instead of standard Zama.
To me: Uses موته (Mota) instead of standard Mala.
To you: Uses توته (Tota) instead of standard Tata.
From me: Uses رونه (Rona) instead of standard Rana.
We: Uses موخ (Moxh) instead of standard Mung.

2. The "Noon Ghunna" (ں) Rule (Crucial!)

Verbs (First Person): When saying "I do something", the verb ends in a nasal Noon Ghunna (ں) instead of the standard "M" (م).
- I do: کاوں (Kaon)
- I wash: وینزاں (Wenzan)
- I write: لېکاں (Lekan)
- I will come: روساں (Rosan)
The "I am" Copula: Standard Pashto uses یم (Yam). Khattak uses یں (Yay + Noon Ghunna). Example: Za khwash یں (I am happy).
Nouns: Nouns do not use the Noon Ghunna. They keep the hard "N" (ن). Example: Asmon (اسمون), Baron (بارون), Pashtun (پښتون).

3. Tenses & Verbs

Future Tense: Uses بو (Bo) instead of standard به (Ba). Example: Za bo sabo char kaon (I will do work tomorrow).
"To Say/Tell": Uses یای (Yai) for "he says" and ویاوں (Wayaon) for "I will tell".
"To Read": Uses یای (Yai) instead of standard Lwali. Example: Agha kitob yai (He is reading a book).

4. Phonetics & Vowel Shifts

The "O" Shift: Many standard "A" sounds shift to a deep "O" sound.
- Asmaan ➡️ Asmon (اسمون)
- Baraan ➡️ Baron (بارون)
- Bazaar ➡️ Bazor (بازور)
- Maakham ➡️ Mashom (ماښوم)
- Yaada ➡️ Yoda (یوده)
Masculine Singular Nouns: Words that end in "i" in standard Pashto often end in "a" in Khattak.
- Gadi (Car) ➡️ Gada (ګاډه)
- Kanri (Stone) ➡️ Konrha (کاڼه)
- Halak/Kakai (Boy) ➡️ Orhka (اوړکه)

5. Unique Khattak Vocabulary

Family: Grandmother = انیو (Anyo)
People: Boy = اوړکه (Orhka), Girl = اوړکې (Orhkay)
Animals: Snake = منګور (Mangor)
Nature: Day = ورېز (Wrez)
Actions: Stand up = چګ شه (Chag sha), Flying = نغېږي (Neghezxhi), Crying = ژوړې (Zorhay)
Adjectives: Big = استر (Astr), Bad/Dirty = کسو (Kso), Tired (Singular) = ستړه (Starha)
Items: Tea leaves = پتې (Pathay)

💻 Installation

markdown
pip install transformers accelerate torch peft

For faster inference (Recommended):

markdown
pip install unsloth

🚀 How to Use

This model is an adapter. To use it, load it alongside the base model.

✅ Method 1 — Unsloth (Recommended - 2x Faster)

Best for: Speed-optimized usage, Colab, A100/H100/T4

markdown
from unsloth import FastLanguageModel
import torch

model_name = "Muhammad-Zubair796/Khatta-ka"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name     = model_name,
    max_seq_length = 2048,
    dtype          = None,
    load_in_4bit   = True,
)
FastLanguageModel.for_inference(model)

alpaca_prompt = """Below is an instruction. Write a detailed response in Pashto.

### Instruction:
{}

### Response:
"""

inputs = tokenizer(
    [
        alpaca_prompt.format("Where did the boy go?", "")
    ], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)
response = tokenizer.batch_decode(outputs, skip_special_tokens = True)[0]

khattak_output = response.split("### Response:\n")[-1].strip()
print(khattak_output)

✅ Method 2 — Standard Transformers & PEFT

Best for: Standard production environments without Unsloth

markdown
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_name = "junaid008/qehwa-pashto-llm"
adapter_name = "Muhammad-Zubair796/Khatta-ka"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype = torch.bfloat16,
    device_map  = "auto",
)

# Load the Khattak Adapter
model = PeftModel.from_pretrained(base_model, adapter_name)

alpaca_prompt = """Below is an instruction. Write a detailed response in Pashto.

### Instruction:
{}

### Response:
"""

def generate(prompt):
    inputs = tokenizer(
        alpaca_prompt.format(prompt, ""),
        return_tensors = "pt",
    ).to("cuda")

    outputs = model.generate(
        **inputs,
        max_new_tokens     = 64,
        temperature        = 0.7,
        do_sample          = True,
        pad_token_id       = tokenizer.eos_token_id,
    )

    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("### Response:\n")[-1].strip()

print(generate("Where did the boy go?"))

⚙️ Hardware Requirements

Table with columns: Method, VRAM, Speed
Method	VRAM	Speed
bfloat16 full	16GB+	✅ Fast
4-bit quantized	8GB+	✅ Good
Unsloth (4-bit)	8GB+	🔥 2x Faster

📊 Training Details

This model was trained using Unsloth for optimized, memory-efficient fine-tuning by Muhammad Zubair.

Table with columns: Parameter, Value
Parameter	Value
Base model	`junaid008/qehwa-pashto-llm`
Training Framework	Unsloth + TRL (SFTTrainer)
Epochs	4
Sequence length	2,048 tokens
LoRA Rank (r)	16
LoRA Alpha	16
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

🗣️ Example Outputs

English input:

Prompt: I am doing nothing, just sitting. Output: زه ایس نه کاوں، بس نوست یں

English input:

Prompt: Where did the boy go? Output: اوړکه چےتا لوړو؟

English input:

Prompt: The weather is very beautiful today, it is raining. Output: نن موسم ډېر شوشته دا، بارون وریږي

📜 License

Apache 2.0 — free to use, modify, and distribute with attribution.

🤝 Citation

markdown
@misc{khattaka-pashto-llm,
  author    = {Muhammad Zubair},
  title     = {Khatta-ka AI: Khattak Pashto LLM},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Muhammad-Zubair796/Khatta-ka}
}