Highlights
- Multimodal. Understands images (and video) as well as text, and answers in Egyptian Arabic. Vision capability comes from the Qwen3.5-2B vision-language foundation (see the note in §5).
- Native Egyptian Arabic. Natural, fluent dialect in conversation, not Modern Standard Arabic dressed up with a few colloquial words. Substantially stronger than the base model on Egyptian Arabic.
- Direct answers. Replies get to the point without filler, preambles or padding, so you spend fewer tokens per useful answer.
- Stable generation. The supervised fine-tuning specifically addresses repetition and looping behavior.
- Calibrated honesty. It does not invent facts, phone numbers, prices or sources, and it does not claim live data, internet access or memory across chats. For volatile facts it points to the official source (راجع المصدر الرسمي).
- Reliable identity. It knows who built it and what it is built on, and does not claim to be another vendor's model.
- Long context. 262,144-token native context window, inherited from the base model.
- Tool calling and coding. Structured tool calls that treat tool, file and web content as data rather than instructions, plus clear coding explanations.
- Open weights. Released in
safetensors format on Hugging Face under Apache-2.0.
The Sovythos family · عيلة Sovythos
Sovythos is an open-source family of language models created by Mahmoud Yasser (محمود ياسر), an Egyptian developer. Each model in the family has its own name and training method.
Table with columns: Model, Type| Model | Type |
|---|
| SYTHS-1.0 (this model) | Supervised fine-tune of Qwen/Qwen3.5-2B on handwritten Egyptian-Arabic data |
Sovythos-66M-Base | Decoder-only Transformer trained from scratch (Sovereign AI Project) |
SYTHS-1.0 is not trained from scratch. It builds on the official Qwen3.5-2B. The "from scratch" description applies only to Sovythos-66M-Base.
1. Overview
SYTHS-1.0 (SYTHS is the model; Sovythos is the family it belongs to) is an open-source multimodal model that takes text and images (and video) as input, produces text, and speaks Egyptian Arabic, with English and Franco-Arabic support. It is a supervised fine-tune (SFT) of the official Qwen/Qwen3.5-2B, trained entirely on handwritten data.
Design goals:
- Natural Egyptian-dialect conversation, including about images.
- Concise, direct responses with no wasted tokens.
- Calibrated honesty and a consistent identity.
- Coding tutoring.
- Reliable tool calling.
- Polite, safe refusals, and confirmation before destructive or sending actions.
2. Model details
Table with columns: Item, Value| Item | Value |
|---|
| Model name | SYTHS-1.0 |
| Family | Sovythos |
| Hugging Face repo | Sovythos/SYTHS-1.0 |
| Base model | Qwen/Qwen3.5-2B (official Qwen release) |
| Fine-tuning method | Supervised fine-tuning (SFT), single stage, Unsloth |
| Weights format | safetensors |
| Type | Causal language model with vision encoder |
| Modalities | Input: text, image, video · Output: text |
| Parameters | ~2B |
| Context window | 262,144 tokens (native) |
| Languages | Egyptian Arabic (primary), English, Franco-Arabic |
| Thinking mode | Off (enable_thinking=False) in training and evaluation |
| License | Apache-2.0 |
| Creator | Mahmoud Yasser (محمود ياسر) |
3. Training procedure
Table with columns: Hyperparameter, Value| Hyperparameter | Value |
|---|
| Base | Qwen/Qwen3.5-2B (16-bit load) |
| LoRA | r = 64, targets q/k/v/o/gate/up/down proj |
| Epochs | 3 |
| Learning rate | 1e-4, cosine schedule |
| Optimizer | adamw_8bit |
| Batch | 1 × grad-accum 16 |
| Max sequence length | 2048 |
| Validation split | 3% |
| Loss | Assistant responses only (train_on_responses_only) |
| Seed | 3407 |
Training aborts automatically if response-only masking is not active (trainable-token fraction must stay between 2% and 98%).
4. Evaluation
SYTHS-1.0 is compared only against its own base model, Qwen3.5-2B, under identical conditions: same prompts, same chat template, greedy decoding, thinking off.
4.1 Internal side-by-side (86 prompts)
The comparison covers identity, honesty, Egyptian-dialect conversation, reasoning, math, and coding prompts.
Table with columns: Metric, Qwen3.5-2B (base), SYTHS-1.0| Metric | Qwen3.5-2B (base) | SYTHS-1.0 |
|---|
| Average response length (tokens) | 288 | 132 |
SYTHS-1.0 answers in less than half the tokens on average. In the same comparison, it answers identity questions correctly (Sovythos, Mahmoud Yasser) where the base model identifies itself as Qwen or, in some cases, as another vendor's model. It also keeps the dialect consistent where the base model drifts into Modern Standard Arabic.
This is an internal prompt suite, not a standardized public benchmark.
4.2 Reference: official base-model results
For reference, these are the official numbers for Qwen/Qwen3.5-2B in Instruct (non-thinking) mode, as published on its model card. They are Qwen's own reported results, not re-run by us.
Table with columns: Benchmark, Qwen3.5-2B (official)| Benchmark | Qwen3.5-2B (official) |
|---|
| MMLU-Pro | 55.3 |
| MMLU-Redux | 69.2 |
| C-Eval | 65.2 |
| SuperGPQA | 30.4 |
| IFEval | 61.2 |
| MMMLU | 56.9 |
This fine-tune targets behavior, dialect and style, not general-knowledge gains. Expect general-knowledge performance to stay close to the base model.
5. Intended use and limitations
Intended use: Egyptian-Arabic assistance (text and image questions), coding explanations, and tool-calling on a small open model. At this model size, it is best suited to prototyping, task-specific fine-tuning, and research and development.
Limitations:
- A 2B-class model can still make factual and reasoning errors. Verify important outputs.
- It has no live data, no internet access and no memory across chats.
- Medical, legal and financial answers are general information, not professional advice.
- Trained on a focused handwritten dataset, so coverage of niche domains is limited.
- Vision: the fine-tuning data is text-only. Image and video understanding is inherited from the Qwen3.5-2B vision-language foundation, and the Egyptian-Arabic style and behavior carry over to visual questions. Vision quality was not separately fine-tuned or benchmarked for SYTHS-1.0. For reference, see the official Qwen3.5-2B vision benchmarks.
- Output language and dialect can drift on very long generations, as with any small model.
- Standardized public benchmark results for SYTHS-1.0 itself have not been published yet.
6. Context window and token limits
Table with columns: Item, SYTHS-1.0| Item | SYTHS-1.0 |
|---|
| Context window (input + output combined) | 262,144 tokens |
| Extended context | Up to 1,010,000 tokens |
| Recommended max output length | 32,768 tokens for most queries |
| Max output for very complex problems | 81,920 tokens |
- If you hit out-of-memory errors, reduce the context window (
--max-model-len in vLLM, --context-length in SGLang).
- SYTHS-1.0 is a ~2B-parameter model. Quality on very long inputs and very long generations is lower than on short ones, so test your own workload before relying on the full window.
- For long video understanding, set
longest_edge to 469762048 in video_preprocessor_config.json (about 224k video tokens).
7. Recommended settings
SYTHS-1.0 is trained and evaluated in non-thinking mode (enable_thinking=False). The /think and /nothink soft switches are not supported.
Recommended sampling parameters for non-thinking mode:
Table with columns: Task, temperature, top_p, top_k, min_p, presence_penalty, repetition_penalty| Task | temperature | top_p | top_k | min_p | presence_penalty | repetition_penalty |
|---|
| Text tasks | 1.0 | 1.0 | 20 | 0.0 | 2.0 | 1.0 |
| Vision-language tasks | 0.7 | 0.8 | 20 | 0.0 | 1.5 | 1.0 |
presence_penalty can be tuned between 0 and 2. Higher values reduce repetition but can occasionally cause language mixing and a small quality drop. SYTHS-1.0 is fine-tuned to avoid repetition, so a lower value is often enough. Tune it for your use case.
- For short, deterministic answers (for example, tests and evaluation), use greedy decoding.
- In multi-turn conversations, keep only the final answer in the history, without any thinking content. The provided chat template already does this.
- Support for sampling parameters varies by inference framework.
8. Usage
Text (Transformers)
import torchfrom transformers import AutoModelForMultimodalLM, AutoProcessor repo = "Sovythos/SYTHS-1.0"processor = AutoProcessor.from_pretrained(repo)model = AutoModelForMultimodalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto") msgs = [{"role": "user", "content": [{"type": "text", "text": "اشرحلي الـ list comprehension في بايثون"}]}]inputs = processor.apply_chat_template( msgs, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=False,).to(model.device) out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=1.0, top_p=1.0, top_k=20)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Image + text
msgs = [{ "role": "user", "content": [ {"type": "image", "url": "https://example.com/photo.jpg"}, {"type": "text", "text": "إيه اللي في الصورة دي؟"}, ],}]inputs = processor.apply_chat_template( msgs, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=False,).to(model.device) out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7, top_p=0.8, top_k=20)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Serving (OpenAI-compatible API)
SYTHS-1.0 needs a recent version of the serving framework. Install vLLM or SGLang from the main branch, or the latest release that supports the Qwen3.5 architecture.
# vLLMvllm serve Sovythos/SYTHS-1.0 --port 8000 --max-model-len 262144 # vLLM with tool callingvllm serve Sovythos/SYTHS-1.0 --port 8000 --max-model-len 262144 \ --enable-auto-tool-choice --tool-call-parser qwen3_coder # vLLM, text-only (skips the vision encoder to free memory for KV cache)vllm serve Sovythos/SYTHS-1.0 --port 8000 --max-model-len 262144 --language-model-only # SGLangpython -m sglang.launch_server --model-path Sovythos/SYTHS-1.0 --port 8000 --context-length 262144
9. About the creator · عن المؤسس
Mahmoud Yasser (محمود ياسر) is an Egyptian developer and the founder of Sovythos. He wrote the entire SYTHS-1.0 training set by hand, designed the training and evaluation pipeline, and builds the SYTHS platform, a Sovythos-family product that gives Arabic speakers a coding and chat assistant.
10. Credits
Built on Qwen3.5 by the Qwen Team (Alibaba Cloud). Trained with Unsloth. SYTHS-1.0 is an independent project, not affiliated with or endorsed by the Qwen Team.
ملخص بالعربي
SYTHS-1.0 موديل مصري مفتوح المصدر من عيلة Sovythos وصناعة محمود ياسر، متعدد الوسائط (رؤية + كتابة)، ومتدرّب (SFT) على موديل Qwen/Qwen3.5-2B الرسمي، على داتا مكتوبة يدويًا بالكامل (من غير أي داتا خارجية ولا مولّد).
أهم المميزات:
- لهجة مصرية طبيعية وأقوى بكتير من الموديل الأساسي.
- كلام دغري من غير رغي ولا تضييع توكنز.
- توليد مستقر من غير تكرار.
- صراحة وعدم تأليف، وهوية ثابتة.
- متعدد الوسائط: بيفهم الصور والفيديو والنص ويرد بالمصري.
- نافذة سياق 262,144 توكن (وتتمدد لحد 1,010,000).
- أقصى طول رد موصى بيه: 32,768 توكن لمعظم الأسئلة، و81,920 للمسائل المعقدة جدًا.
- شرح برمجة واستدعاء أدوات بشكل سليم.
الأوزان متاحة بصيغة safetensors على Hugging Face: Sovythos/SYTHS-1.0.
ملاحظة عن الاسم: اسم الموديل العام SYTHS-1.0. داتا التدريب بتعرّف الموديل باسم عيلته Sovythos (وده مقصود)، فممكن يقول إنه "Sovythos" لما يتسأل عن هويته.