Kenvenx

Arabian-Agent-Qwen3-0.6B

README

License: apache-2.0

✨ What is Arabian-Agent?

Arabian-Agent is a small, Arabic-first agent model designed to choose an available tool and produce a structured call rather than pretending it has already completed the task. It is a LoRA fine-tune of Qwen/Qwen3-0.6B, merged into the base weights for convenient inference and re-quantization.

Use it as the tool-routing component inside an agent runtime—not as a replacement for the runtime, permission system, or tool execution layer.

📊 Evaluation snapshot

Results from the project's frozen 354-example test set, using greedy decoding and the same evaluation harness across versions.

Table with columns: Metric, Base Qwen3-0.6B, Arabian-Agent v4
Metric	Base Qwen3-0.6B	Arabian-Agent v4
Tool selection accuracy	92.17%	98.16%
Exact argument match	49.31%	82.03%
Valid output rate	98.31%	99.72%
Arabic tool selection	92.54%	100.00%
Arabic exact argument match	49.25%	83.58%
English exact argument match	49.33%	81.33%
False completions	1	0

These are project-reported results on a curated test battery, not a guarantee of production performance. See the limitations below.

🚀 Quick start

Transformers

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Kenvenx/Arabian-Agent-Qwen3-0.6B"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(    model_id,    torch_dtype=torch.bfloat16,).eval() messages = [    {        "role": "system",        "content": """You are an AI agent. Use only tools listed below.Available tools:search(query: str) -> list — search the web. When a tool is needed, output:<tool>{"name":"search","arguments":{"query":"..."}}</tool>Do not invent tool results. Answer in the user's language."""    },    {"role": "user", "content": "ابحث لي عن أخبار الذكاء الاصطناعي في السعودية"},]inputs = tokenizer.apply_chat_template(    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")with torch.no_grad():    output = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Expected shape (illustrative):

text

<tool>{"name":"search","arguments":{"query":"أخبار الذكاء الاصطناعي في السعودية"}}</tool>

llama.cpp / Ollama / LM Studio

For local inference, choose a quantization in the GGUF repository.

bash

# llama.cppllama-cli -hf Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M # OpenAI-compatible server for an agent runtimellama-server -hf Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M --ctx-size 8192 # Ollamaollama run hf.co/Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M

Suggested decoding: greedy decoding (do_sample=False) or temperature ≤ 0.2 for tool calls. If the runtime exposes Qwen3 thinking controls, disabling thinking is a sensible starting point for this direct-output format.

🧩 Tool-call protocol

The model was trained to emit this compact format when a tool is needed:

text

<tool>{"name":"tool_name","arguments":{"key":"value"}}</tool>

Include the available tool names, argument schemas, and short descriptions in the system prompt.
Parse the output and validate it against a strict schema before execution.
Return the actual tool result to the model; never treat a proposed call as proof that the task succeeded.
If no tool is needed, the model may answer in plain text.

🏗️ Training summary

Table with columns: Item, Details
Item	Details
Base	Qwen/Qwen3-0.6B
Fine-tuning	LoRA r=16, α=32, then merged into base weights
Training examples	17,355
Supervised tokens	≈2.3 million
Training	1 epoch / 4,338 steps
Compute	2× NVIDIA Tesla T4 on Kaggle
Peak VRAM	8.48 GB
Sequence length	2,048 tokens
Seed	13
License	Apache-2.0

⚠️ Known limitations

Multi-step follow-through needs more work: similarity-to-reference on 57 post-tool-result examples was 0.4939 in the reported evaluation.
Open-ended Arabic chat is not its strength: distinct-4 was 0.323 on 40 Arabic no-tool answers, versus 0.713 for the base model.
Modern Standard Arabic is better represented than dialects and heavy code-switching.
A 0.6B model can struggle with long, ambiguous, or reasoning-heavy requests.
The evaluation does not establish robustness against jailbreaks, factuality, coding, math, or production traffic.

🔐 Safety and deployment

The model proposes tool calls; your application remains responsible for safety.

Validate tool names and argument types against an allowlist/schema.
Enforce user permissions and least privilege outside the model.
Require confirmation for consequential or irreversible actions.
Apply timeouts, logging, rate limits, and error handling.
Never execute arbitrary model text as shell commands or code.

📚 Model details

Architecture: Qwen3ForCausalLM; approximately 606M parameters.
Output: compact <tool>…</tool> calls or ordinary assistant text.
Fine-tuning data: mixed English tool-use trajectories, Arabic tool calls, and Arabic assistant responses. Review each source's license before redistribution.
Quantized variants: Q4_K_M, Q5_K_M, Q6_K, Q8_0 and Q3_K_M.

🙏 Acknowledgements

Built on Qwen3, with training data including Salesforce xLAM, PinchBench Clawd, and Arabic Alpaca. GGUF conversion uses llama.cpp.

Made for lightweight agents. Built with Arabic in mind.
If you use it, please share your evaluation setup and failure cases.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Kenvenx

Model Tree

Base

Qwen/Qwen3-0.6B

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

✨ What is Arabian-Agent?

Use it as the tool-routing component inside an agent runtime—not as a replacement for the runtime, permission system, or tool execution layer.

📊 Evaluation snapshot

Results from the project's frozen 354-example test set, using greedy decoding and the same evaluation harness across versions.

Table with columns: Metric, Base Qwen3-0.6B, Arabian-Agent v4
Metric	Base Qwen3-0.6B	Arabian-Agent v4
Tool selection accuracy	92.17%	98.16%
Exact argument match	49.31%	82.03%
Valid output rate	98.31%	99.72%
Arabic tool selection	92.54%	100.00%
Arabic exact argument match	49.25%	83.58%
English exact argument match	49.33%	81.33%
False completions	1	0

These are project-reported results on a curated test battery, not a guarantee of production performance. See the limitations below.

🚀 Quick start

Transformers

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Kenvenx/Arabian-Agent-Qwen3-0.6B"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(    model_id,    torch_dtype=torch.bfloat16,).eval() messages = [    {        "role": "system",        "content": """You are an AI agent. Use only tools listed below.Available tools:search(query: str) -> list — search the web. When a tool is needed, output:<tool>{"name":"search","arguments":{"query":"..."}}</tool>Do not invent tool results. Answer in the user's language."""    },    {"role": "user", "content": "ابحث لي عن أخبار الذكاء الاصطناعي في السعودية"},]inputs = tokenizer.apply_chat_template(    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")with torch.no_grad():    output = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Kenvenx/Arabian-Agent-Qwen3-0.6B"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(    model_id,    torch_dtype=torch.bfloat16,).eval() messages = [    {        "role": "system",        "content": """You are an AI agent. Use only tools listed below.Available tools:search(query: str) -> list — search the web. When a tool is needed, output:<tool>{"name":"search","arguments":{"query":"..."}}</tool>Do not invent tool results. Answer in the user's language."""    },    {"role": "user", "content": "ابحث لي عن أخبار الذكاء الاصطناعي في السعودية"},]inputs = tokenizer.apply_chat_template(    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")with torch.no_grad():    output = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Expected shape (illustrative):

text

<tool>{"name":"search","arguments":{"query":"أخبار الذكاء الاصطناعي في السعودية"}}</tool>

<tool>{"name":"search","arguments":{"query":"أخبار الذكاء الاصطناعي في السعودية"}}</tool>

llama.cpp / Ollama / LM Studio

For local inference, choose a quantization in the GGUF repository.

bash

# llama.cppllama-cli -hf Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M # OpenAI-compatible server for an agent runtimellama-server -hf Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M --ctx-size 8192 # Ollamaollama run hf.co/Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M

# llama.cppllama-cli -hf Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M # OpenAI-compatible server for an agent runtimellama-server -hf Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M --ctx-size 8192 # Ollamaollama run hf.co/Kenvenx/Arabian-Agent-Qwen3-0.6B-GGUF:Q4_K_M

🧩 Tool-call protocol

The model was trained to emit this compact format when a tool is needed:

text

<tool>{"name":"tool_name","arguments":{"key":"value"}}</tool>

Include the available tool names, argument schemas, and short descriptions in the system prompt.
Parse the output and validate it against a strict schema before execution.
Return the actual tool result to the model; never treat a proposed call as proof that the task succeeded.
If no tool is needed, the model may answer in plain text.

🏗️ Training summary

Table with columns: Item, Details
Item	Details
Base	Qwen/Qwen3-0.6B
Fine-tuning	LoRA r=16, α=32, then merged into base weights
Training examples	17,355
Supervised tokens	≈2.3 million
Training	1 epoch / 4,338 steps
Compute	2× NVIDIA Tesla T4 on Kaggle
Peak VRAM	8.48 GB
Sequence length	2,048 tokens
Seed	13
License	Apache-2.0

⚠️ Known limitations

Multi-step follow-through needs more work: similarity-to-reference on 57 post-tool-result examples was 0.4939 in the reported evaluation.
Open-ended Arabic chat is not its strength: distinct-4 was 0.323 on 40 Arabic no-tool answers, versus 0.713 for the base model.
Modern Standard Arabic is better represented than dialects and heavy code-switching.
A 0.6B model can struggle with long, ambiguous, or reasoning-heavy requests.
The evaluation does not establish robustness against jailbreaks, factuality, coding, math, or production traffic.

🔐 Safety and deployment

The model proposes tool calls; your application remains responsible for safety.

Validate tool names and argument types against an allowlist/schema.
Enforce user permissions and least privilege outside the model.
Require confirmation for consequential or irreversible actions.
Apply timeouts, logging, rate limits, and error handling.
Never execute arbitrary model text as shell commands or code.

📚 Model details

Architecture: Qwen3ForCausalLM; approximately 606M parameters.
Output: compact <tool>…</tool> calls or ordinary assistant text.
Fine-tuning data: mixed English tool-use trajectories, Arabic tool calls, and Arabic assistant responses. Review each source's license before redistribution.
Quantized variants: Q4_K_M, Q5_K_M, Q6_K, Q8_0 and Q3_K_M.

🙏 Acknowledgements

Built on Qwen3, with training data including Salesforce xLAM, PinchBench Clawd, and Arabic Alpaca. GGUF conversion uses llama.cpp.

Made for lightweight agents. Built with Arabic in mind.
If you use it, please share your evaluation setup and failure cases.