Model Details
- Base model: Qwen/Qwen3-4B (Apache 2.0)
- Method: QLoRA — 4-bit NF4 quantization, LoRA r=16, alpha=32, dropout 0.05, all attention + MLP projections
- License: Apache 2.0 (inherited from base; this adapter is a derivative work with attribution)
- Trained by: Menasaat | Virtual Platforms LLC on Kaggle free-tier GPUs (Tesla T4, fp16)
Training Data
menasaat/oman-wikipedia-corpus —
3,926 instruction pairs (60 held out for eval):
"Tell me about X" → lead section, and "More details about X" → article passages, in Arabic and English.
Training Results
Table with columns: Metric, Value| Metric | Value |
|---|
| Training time | 140.1 minutes (1× Tesla T4) |
| Training loss | 6.33 → 3.91 |
| Eval loss (held-out) | 4.00 |
Evaluation: Base vs Fine-tuned (2026-09-21)
Table with columns: Metric, Base Qwen3-4B, menasaat fine-tune, Change| Metric | Base Qwen3-4B | menasaat fine-tune | Change |
|---|
| Mean NLL, 60 held-out Oman pairs (AR/EN) | 3.14 | 2.01 | −36.1% |
| Train loss (3,926 pairs) | — | 6.33 → 3.91 | — |
Full side-by-side answers (20 questions, AR+EN) in eval_results.json.
Example — "أخبرني عن ولاية صلالة." (Tell me about Salalah):
the base model located Salalah in Malaysia; the fine-tune correctly places it in
the Sultanate of Oman. Generations remain imperfect (verify details against
primary sources), but Oman grounding is materially improved.
Usage
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B")model = PeftModel.from_pretrained(base, "menasaat/qwen3-4b-oman-qlora")tok = AutoTokenizer.from_pretrained("menasaat/qwen3-4b-oman-qlora")
Limitations
Pilot run (2 epochs). Answers are grounded in Wikipedia text about Oman and
reflect its coverage and biases; verify facts against primary sources.
About the publisher | عن الناشر
Menasaat | Virtual Platforms LLC is an enterprise AI company serving Oman, the GCC, and the wider MENA region. We build Arabic-first AI solutions: open data about the Sultanate of Oman, fine-tuned Arabic/English large language models (LLMs), retrieval (RAG) systems, and production AI for enterprises and the public sector. Explore our work at menasaat.com and on Hugging Face.
منصات افتراضية ش.م.م (Menasaat) هي شركة ذكاء اصطناعي مؤسسي تخدم سلطنة عُمان ودول مجلس التعاون الخليجي ومنطقة الشرق الأوسط وشمال أفريقيا. نبني حلول ذكاء اصطناعي عربية أولاً: بيانات مفتوحة عن سلطنة عُمان، ونماذج لغوية كبيرة (LLM) مُدرَّبة بالعربية والإنجليزية، وأنظمة استرجاع معزّز بالتوليد (RAG)، وحلول ذكاء اصطناعي إنتاجية للمؤسسات والقطاع العام.