GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
tokenaii
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
tokenaii
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Item | Value |
|---|---|
| Model family | Horus Taleeq |
| Model type | Arabic-only Small Language Model (SLM), decoder-only causal Transformer |
| Approximate parameters | exactly 199,916,160 (about 199.92M) |
| Primary language | Arabic |
| Intended text | Modern Standard Arabic and Arabic dialectal text |
| Egyptian Arabic | Targeted in the training program; not independently certified by this base checkpoint |
| Vocabulary | 128,000-token Arabic-first SentencePiece tokenizer |
| Hidden size | 640 |
| Transformer layers | 24 |
| Attention heads | 10 |
| Key/value heads | 2 (GQA) |
| MLP intermediate size | 2,048 |
| Activation | SiLU/SwiGLU-style Llama MLP |
| Normalization | RMSNorm, epsilon 1e-5 |
| Position encoding | RoPE, theta 1,000,000 |
| Embeddings | Tied input and output embeddings |
| Tensor type | bfloat16 |
| Context configuration | 4,096 tokens; initial pretraining sequences used 2,048 tokens |
The uploaded checkpoint is the completed clean-repair-v1 base checkpoint. The
later canonical-clean continuation is still a separate development run and is
not included in this upload.
This repository is intended for the verified base checkpoint only. The original training ledger records approximately 6.0B tokenizer IDs across earlier training streams, but those historical streams are not yet a fully reproducible public corpus. A separate canonical-clean continuation is being evaluated and must not be represented as complete until its checkpoint and data manifest are published.
The current release therefore makes no claim of finished chat quality, factual benchmark leadership, or production readiness. Earlier conversation adapters are not part of this base repository.
The reproducibility baseline uses fused AdamW with betas (0.9, 0.95), epsilon
1e-8, weight decay 0.1, gradient clipping 1.0, CUDA bfloat16 autocast, and
activation checkpointing. STAM is not the optimizer used for this verified base
checkpoint.
This base model is ready for continued pretraining, instruction tuning, identity tuning, and downstream Arabic applications. It is not a safety-tuned assistant and should not be used as a production chatbot without additional alignment, safety, and quality evaluation.
The planned base release contains the model configuration, bfloat16 weights, generation defaults, the matching 128K base tokenizer, the MIT license, and this model card. The role-special tokenizer created for future chat SFT is deliberately not presented as the tokenizer used by this base pretraining release. No SFT adapter, teacher outputs, private credentials, or raw training corpus is included.
python
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "tokenaii/Horus-Taleeq-0.2-base"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained( repo, torch_dtype="auto", device_map="auto",) prompt = "اكتب فقرة قصيرة عن أهمية القراءة."inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=120, do_sample=False)print(tokenizer.decode(outputs[0], skip_special_tokens=True))bibtex
@misc{tokenai_horus_taleeq_02_base_2026, title = {Horus Taleeq 0.2 Base}, author = {Assem Sabry and TokenAI}, year = {2026}, publisher = {Hugging Face}, howpublished = {\\url{https://huggingface.co/tokenaii/Horus-Taleeq-0.2-base}}, note = {Arabic-first decoder-only base language model}}This model is released under the MIT License. Users remain responsible for complying with applicable laws, third-party data rights, and the terms of any data sources used in downstream training or evaluation.
from transformers import AutoModelForCausalLM, AutoTokenizer repo = "tokenaii/Horus-Taleeq-0.2-base"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained( repo, torch_dtype="auto", device_map="auto",) prompt = "اكتب فقرة قصيرة عن أهمية القراءة."inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=120, do_sample=False)print(tokenizer.decode(outputs[0], skip_special_tokens=True))@misc{tokenai_horus_taleeq_02_base_2026, title = {Horus Taleeq 0.2 Base}, author = {Assem Sabry and TokenAI}, year = {2026}, publisher = {Hugging Face}, howpublished = {\\url{https://huggingface.co/tokenaii/Horus-Taleeq-0.2-base}}, note = {Arabic-first decoder-only base language model}}