GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nassimjp
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
nassimjp
Model Tree
Qwen/Qwen3-VL-2B-Thinking
Input Modalities
Output Modalities
Supported Functionality
Qwen/Qwen3-VL-2B-Thinkingnassimjp/Qwen3-VL-2B-Thinking-Pashto-Zi151,669151,700312048torch.bfloat16The original embedding matrix had the shape:
text
(151936, 2048)After tokenizer extension and resizing, the language-model embedding matrix has the shape:
text
(151700, 2048)The input and output embeddings share the same storage (tied embeddings).
The vision tower of Qwen3-VL is not vocab-indexed and was not touched.
Qwen3-VL already ships with very broad Arabic-script coverage. Of the 175 unique atoms audited, 144 were already single tokens — including all Pashto-specific letters, most Sindhi letters, and most Balochi vowels.
But 31 atoms were still missing and split into byte-fallback pairs. For Pashto, Sindhi, Balochi, and Persian/Urdu numeric text, this is a real tokenization inefficiency.
Examples of atoms that were previously split:
text
ٽڇڏڙڦڻۏ۰۱۲۳۴۵۶۷۸۹These characters previously tokenized as multi-byte fragments such as
[151, 108], [150, 237], etc.
The purpose of this project is to give them their own independent vocabulary entries.
A total of 175 unique atoms were audited (181 raw entries, with intentional duplicates preserved for a separate "push-all" script).
text
Existing single-token atoms : 144Atoms requiring new token : 31The audit found 31 missing atoms requiring new vocabulary entries.
Examples of previously missing characters:
text
ٽڇڏڙڦڻۏAdditional extended Arabic-Indic digits, bidi controls, and harakat were also added.
The original vocabulary contained:
text
151669After adding 31 new tokens:
text
151700| Token | ID |
|---|---|
| ٽ | 151669 |
| ڇ | 151670 |
| ڏ | 151671 |
| ڙ | 151672 |
| ڦ | 151673 |
| ڻ | 151674 |
| ۏ | 151675 |
| ۰ | 151676 |
| ۱ | 151677 |
| ۲ |
Each of the 31 new vocabulary entries receives its own independent embedding row.
The new embeddings were initialized independently with:
text
Initializer range: 0.02Embedding dimension: 2048The forensic audit confirmed:
text
Expected shape: (31, 2048)Actual shape: (31, 2048)Each new token therefore owns an independent 2048-dimensional embedding vector.
An exact duplicate search was performed across the new embedding rows.
Result:
text
NO EXACT DUPLICATE EMBEDDING ROWSA pairwise cosine similarity analysis was also performed.
The maximum cosine similarity between distinct new-token embeddings was:
text
0.075313299894The corresponding pair was:
text
۲ <-> ٔThis confirms that the newly initialized embedding rows are distinct rather than duplicated copies.
text
mean = -0.0000267930std = 0.0322032236norm mean = 1.43841958text
mean = 0.0000779270std = 0.0200125575norm mean = 0.90553617The new-token embeddings were independently initialized and have their own embedding distribution. They are intentionally initialized slightly tighter than the pretrained rows, so that continued pretraining can draw them into the model's existing distribution naturally.
For comparison, the same surgery on LiquidAI/LFM2.5-2.6B added 56 new tokens.
Qwen3-VL-2B-Thinking already covered 25 of those atoms as single tokens,
because the Qwen tokenizer has broader Arabic-script coverage out of the box.
| Atom family | LFM2.5-2.6B | Qwen3-VL-2B-Thinking |
|---|---|---|
| Pashto letters | missing | ✅ already single |
| Sindhi letters | missing | mostly ✅, 6 missing |
| Balochi vowels | missing | mostly ✅, 1 missing |
| Arabic-Indic digits | partially missing | ❌ all 10 missing |
| Harakat | partially missing | mostly ✅, 6 missing |
| Bidi controls |
The final forensic checks confirmed:
text
✓ Input/output embeddings share storage (tied)✓ Every new atom owns an independent embedding row✓ No exact duplicate embedding rows✓ No tokenizer-ID/indexing corruption detected✓ Embedding matrix successfully resized✓ Vision tower untouched (not vocab-indexed)✓ Model successfully saved✓ Tokenizer successfully saved✓ Model successfully loaded again from Hugging FaceAfter uploading the model, it was loaded again from:
text
nassimjp/Qwen3-VL-2B-Thinking-Pashto-ZiThe loaded model reported:
text
Vocab size:151700Embedding shape:torch.Size([151700, 2048])The newly added tokens were also verified after reloading.
Examples:
text
ٽ -> 151669ڇ -> 151670ڏ -> 151671ڙ -> 151672ڦ -> 151673ڻ -> 151674Existing tokens that were already present in the original tokenizer retain their original IDs.
For example:
text
ښ -> 147996ځ -> 148519ږ -> 150389ټ -> 148249ډ -> 148967ړ -> 148871ګ -> 150392These are existing tokenizer entries, not newly added vocabulary entries.
This repository represents a tokenizer and embedding vocabulary extension.
The 31 newly added token embeddings were freshly initialized during the vocabulary surgery.
This does not by itself mean that the model has already learned full Pashto (or Sindhi, Balochi, …) language knowledge for these new atoms.
For the newly introduced tokens to acquire meaningful linguistic representations, further training such as continued pretraining / causal language modeling on multilingual Arabic-script data is required.
In other words:
text
Tokenizer surgery ↓31 new Pashto + Sindhi + Balochi + Arabic-script atoms ↓Independent embedding rows ↓Continued pretraining ↓Learned representationsThis model is intended as an experimental foundation for further Arabic-script language-model development on top of Qwen3-VL.
Potential next steps include:
The repository contains the model, tokenizer, configuration, generation configuration, chat template, and forensic report.
Important files include:
text
config.jsonmodel.safetensorsgeneration_config.jsontokenizer_config.jsontokenizer.jsonvocab.jsonmerges.txtchat_template.jinjapashto_urdu_fresh_embedding_forensics_Qwen_Qwen3-VL-2B-Thinking.jsonThe forensic report records the tokenizer audit, vocabulary additions, embedding checks, and verification results.
This project is based on:
text
Qwen/Qwen3-VL-2B-ThinkingBase model:
Qwen — Qwen3-VL-2B-Thinking
This repository does not claim to reproduce or replace the original base model. It provides an extended tokenizer vocabulary and corresponding resized embedding matrix.
python
from transformers import AutoTokenizer, Qwen3VLForConditionalGenerationmodel_id = "nassimjp/Qwen3-VL-2B-Thinking-Pashto-Zi"tokenizer = AutoTokenizer.from_pretrained(model_id)model = Qwen3VLForConditionalGeneration.from_pretrained( model_id, dtype="auto", device_map="auto")print("Vocabulary:", len(tokenizer))print("Embedding:", model.get_input_embeddings().weight.shape)Expected output:
text
Vocabulary: 151700Embedding: torch.Size([151700, 2048])You can inspect the newly added atoms with:
python
new_atoms = [ # Sindhi "ٽ", "ڇ", "ڏ", "ڙ", "ڦ", "ڻ", # Balochi "ۏ", # Extended Arabic-Indic digits "۰", "۱", "۲", "۳", "۴", "۵", "۶", "۷", "۸", "۹", # Symbol "؉", # Bidi controls "\u200d", "\u200f", "\u202a", "\u202b", "\u202c", "\u202d", "\u202e", # Harakat "ٌ", "ٍ", "ٓ", "ٔ", "ٕ", "ٰ",]for char in new_atoms: token_id = tokenizer.convert_tokens_to_ids(char) print(repr(char), "->", token_id)Expected IDs run from 151669 through 151699 (see the table above).
After the surgery, the following text categories tokenize more efficiently:
| Category | Before | After |
|---|---|---|
Eastern Arabic-Indic digits (۰۱۲۳۴۵۶۷۸۹) | 2 tokens each | 1 token each |
Sindhi consonants ٽ ڇ ڏ ڙ ڦ ڻ | 2 tokens each | 1 token each |
Balochi vowel ۏ | 2 tokens | 1 token |
| Bidi controls | 2 tokens each | 1 token each |
| Select harakat |
Approximate efficiency gains on Pashto, Sindhi, Balochi, and Persian/Urdu numeric text: ~10–20% fewer tokens for text that uses these atoms.
Tokenizer surgery: ✅ Complete
Vocabulary resize: ✅ Complete
Embedding forensic audit: ✅ Passed
Model save: ✅ Complete
Tokenizer save: ✅ Complete
Hugging Face upload: ✅ Complete
Reload verification: ✅ Complete
Continued pretraining: ⏳ Not performed as part of this surgery
SFT / instruction tuning: ⏳ Not performed as part of this surgery
A complete forensic report was generated during the surgery:
text
pashto_urdu_fresh_embedding_forensics_Qwen_Qwen3-VL-2B-Thinking.jsonThe report contains:
nassimjp
Pashto AI / Pashto NLP experiments
Please refer to the license of the original base model and repository configuration before redistributing modified model weights.
Qwen3-VL-2B-Thinking-Pashto-Zi
A focused tokenizer surgery on top of an already multilingual base:
text
Qwen/Qwen3-VL-2B-Thinking +31 new Pashto + Sindhi + Balochi + Arabic-script atoms ↓151,669 → 151,700 vocabulary ↓Multilingual Arabic-script tokenizer foundation +Vision tower preserved intactThe tokenizer now has dedicated vocabulary entries for the missing Pashto, Sindhi, Balochi, and related Arabic-script characters that Qwen3-VL previously split into byte fragments.
text
---nassimjp/LFM2.5-2.6B-Pashto-Zi-b — same surgery, 56 new tokens, on LiquidAI/LFM2.5-2.6Btext
undefined| 151678 |
| ۳ | 151679 |
| ۴ | 151680 |
| ۵ | 151681 |
| ۶ | 151682 |
| ۷ | 151683 |
| ۸ | 151684 |
| ۹ | 151685 |
| ؉ | 151686 |
| U+200D | 151687 |
| U+200F | 151688 |
| U+202A | 151689 |
| U+202B | 151690 |
| U+202C | 151691 |
| U+202D | 151692 |
| U+202E | 151693 |
| ٌ | 151694 |
| ٍ | 151695 |
| ٓ | 151696 |
| ٔ | 151697 |
| ٕ | 151698 |
| ٰ | 151699 |
| mostly missing |
| mostly missing |
| 2 tokens each |
| 1 token each |