GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
techwithrayoma
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
techwithrayoma
Model Tree
Qwen/Qwen2.5-1.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Held-out benchmark of 375 real Arabic comments, labeled by hand.
| Accuracy | F1 macro | |
|---|---|---|
| Base, zero-shot | 35.5% | 0.248 |
| + this adapter | 78.1% | 0.516 |
| Class | F1 |
|---|---|
| Question | 0.88 |
| Complaint | 0.79 |
| Statement | 0.53 |
| Praise | 0.38 |
| Suggestion | 0.00 |
Suggestion does not work. F1 is 0.00 — five benchmark examples, so the model never
learns the class. If suggestions matter to you, this adapter isn't ready for it.
Praise at 0.38 is close to a coin flip.
Training labels came from gpt-4o-mini, not humans. The benchmark was annotated by
hand, which is why the failures above are visible rather than hidden — but the ceiling on
the working classes is still the teacher's judgment.
Domain: YouTube comments about one fintech product, Saudi and Gulf dialect. Other domains, dialects, and Modern Standard Arabic are untested.
python
from transformers import AutoModelForCausalLM, AutoTokenizer, pipelinefrom peft import PeftModelimport torch, jsonADAPTER = "techwithrayoma/barqpulse-v1-qwen2.5-1.5b-lora"SYSTEM = ("You are a strict text classification system.\n""Classify the comment into exactly one of these labels:\n""Question, Complaint, Statement, Praise, Suggestion.\n""Return ONLY valid JSON in this exact format:\n"'{"predicted_intent": "<label>"}\n'"Do not output anything else.")tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", torch_dtype=torch.float16, device_map="auto")pipe = pipeline("text-generation", model=PeftModel.from_pretrained(base, ADAPTER),tokenizer=tok, max_new_tokens=32, do_sample=False)def classify(comment):prompt = tok.apply_chat_template([{"role": "system", "content": SYSTEM},{"role": "user", "content": f"Comment:\n{comment}"}],tokenize=False, add_generation_prompt=True)out = pipe(prompt)[0]["generated_text"][len(prompt):]return json.loads(out[out.find("{"):out.rfind("}") + 1])["predicted_intent"]classify("كم المده بعد الطلب لحتى توصل البطاقه") # → "Question"
⚠️ The system prompt is not optional. The model was fine-tuned on this exact format. A different one will score well below the numbers above.
max_new_tokensmust be at least 32 — the JSON needs about 12 tokens, and truncating it breaks every prediction.
| Method | LoRA rank 64, lora_target: all, LLaMA-Factory SFT |
| Schedule | 3 epochs, cosine, lr 1e-4, bf16 |
| Batching | per-device 2 × grad accum 4, cutoff 3500 |
| Data | 921 Arabic YouTube comments, teacher-labeled by gpt-4o-mini |
Triage and analytics on Arabic customer feedback at volume. Not for automated decisions about individual people — two of five classes don't work well enough, and the model hasn't been evaluated for bias across dialects or demographics.
Apache 2.0, following the base model.