GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
themohal
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
themohal
Model Tree
Qwen/Qwen3-8B-Base
Input Modalities
Output Modalities
Supported Functionality
This is a raw language-model completion adapter, not an instruction-following assistant. Given Saraiki text, it continues it fluently — it has not been trained to follow instructions, answer questions conversationally, or hold a dialogue. That behavior is added in Stage 2, which merges this adapter into the base model before training an instruction-following LoRA on top. Most users should start from Stage 2 (or Stage 3, for tool use) rather than this model directly, unless you specifically need raw Saraiki text completion.
python
from unsloth import FastLanguageModelfrom peft import PeftModelfrom transformers import AutoTokenizerimport torchBASE_MODEL = "Qwen/Qwen3-8B-Base"CPT_ADAPTER = "themohal/saraiki-qwen3-8b-cpt"tokenizer = AutoTokenizer.from_pretrained(CPT_ADAPTER)model, _ = FastLanguageModel.from_pretrained(model_name=BASE_MODEL, load_in_4bit=True, dtype=None,)model.resize_token_embeddings(len(tokenizer))model = PeftModel.from_pretrained(model, CPT_ADAPTER)# Plain text completion — no chat template, no instruction-following.prompt = "سرائیکی ادب دی تاریخ اچ"inputs = tokenizer(prompt, return_tensors="pt").to("cuda")outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7, top_p=0.9)print(tokenizer.decode(outputs[0], skip_special_tokens=True))
themohal/saraiki-llm-dataset — a Saraiki text corpus combining cleaned/deduplicated bulk sources (raw sentences, a parallel corpus, newspaper/literature/magazine text) with Gemini-generated long-form passages across varied genres (short stories, essays, descriptive writing, folklore, and more).
This dataset grows daily (roughly once per morning), so this model is retrained continually rather than as a single fixed run — see Continual training below.
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj; base model weights frozen throughout.Because the training dataset grows daily, this model does not use naive resume_from_checkpoint on every run — that assumes a static dataset, which a growing corpus violates (the train/validation split membership and packed-sequence composition change as rows are added). Instead:
data_manifest.json alongside each pushed checkpoint.resume_from_checkpoint (same optimizer state, same LR schedule position) — this is what happens on same-day reruns.This means the adapter is best understood as a continually-improving artifact rather than a single frozen release — total training step count naturally grows session to session as the corpus grows, and each session's evaluation numbers aren't directly comparable to a prior session's (the validation split membership shifts too). Check the commit history / data_manifest.json in the most recent checkpoint folder for what it was actually trained on.
| Repo | Purpose |
|---|---|
themohal/saraiki-qwen-8b-sft | Stage 2 — instruction/assistant SFT LoRA, built on top of this model |
themohal/saraiki-qwen-8b-tool-use | Stage 3 — tool-use/function-calling SFT LoRA |
themohal/saraiki-llm-dataset | Stage 1 training corpus — this model (grows daily) |
Muhammad Farjad Ali Raza
Apache 2.0, inherited from the Qwen3-8B-Base license. Training data combines cleaned real-world text with synthetically generated content; see the dataset card for details.
themohal/saraiki-assistant-sft| Stage 2 training data (grows daily) |
themohal/saraiki-tool-use-sft | Stage 3 training data (grows daily) |