Mixture of Experts (MoE) Architecture
Zenthi-AI functions as an orchestrator that dynamically routes user queries to specialized model experts depending on request characteristics:
- Vision-Language Expert (VLM)
- Model:
riven/smolvlm:latest (~545 MB)
- Purpose: Visual reasoning, image descriptions, GUI analysis, and OCR document queries.
- Document & General Thinking Expert
- Model:
qwen2.5:1.5b-instruct (~986 MB)
- Purpose: General knowledge processing, document reading, semantic summarization, and copy generation.
- Coding & Tool Selection Expert
- Model:
qwen2.5-coder:3b (~1.9 GB)
- Purpose: Code generation, bug debugging, software engineering, and tool execution queries.
Datasets Used in Fine-Tuning
The base Qwen2.5-0.5B-Instruct model was fine-tuned using QLoRA (4-bit quantization) on a curated blend of several high-quality datasets:
- DailyDialog: For clean multi-turn conversational dialogue.
- Persona-Chat: For styling agent persona and contextual roleplay.
- Dialog-bAbI-Tasks: For step-by-step logic, reasoning, and QA tasks.
- CodeParrot-Small: For clean Python implementation samples.
- MiniPile: For general-knowledge facts and topic synthesis.
- TinyStories: For story generation and narrative comprehension.
- HumanEval: For Python code validation and programming accuracy.
Licensing Compliance
This model is fully compliant with the Apache 2.0 License in accordance with its base Qwen2.5 model guidelines. You are free to modify, distribute, and utilize this model for commercial or non-commercial applications.