Highlights
- Even with its base on general benchmarks, ahead on tools: against Qwen3.5-4B on 22 public benchmarks, 2 real wins
(Winogrande, BFCL tool calling), 1 real loss (DROP), 19 ties.
- Tool calling (BFCL): 88.7% vs 85.2% for the base.
- Serbian: ties YugoGPT 7B on its own Serbian LLM Eval (59.0 vs 60.5, within the
margin) at about half the size, ahead of its base (57.9); Belebele Serbian 86.6% vs 84.6%.
- Thinking mode: MATH-500 95.0%, MMLU-Pro 81.0%, GPQA Diamond 62.6%.
- Against other 4B-class models (thinking off): vs Qwen3 4B 2507 8 real wins / 3 losses;
vs Gemma 4 E4B 6 real wins / 3 losses.
- Knows who it is with no system prompt, holds a correct answer under pushback, solves more real agent tasks than its base.
How it was made
- Serbian continued pretraining: ~50M tokens of Serbian (Latin and Cyrillic) and English text → v0.2-base.
- Fine-tune: ~31,000 examples (chat, agents, tool calls, code, instruction following, maths, Serbian,
uncertainty, identity) plus verified step-by-step reasoning traces.
- Blend: 80% fine-tuned weights + 20% Qwen3.5-4B (WiSE-FT). The raw fine-tune was behind its base on
MMLU-Pro, GSM8K and instruction following; the blend wins those back while keeping the new skills.
Benchmarks (thinking off)
Same harness, prompts, settings and question samples for every model; Q4_K_M GGUF on llama.cpp, no system prompt.
✅ / ❌ = a real win / loss against the base (beyond 2 standard errors); unmarked rows are ties. Best per row in bold.
Table with columns: Benchmark, Quata2 4B v0.2, Qwen3.5 4B (base), Qwen3 4B 2507, Gemma 4 E4B| Benchmark | Quata2 4B v0.2 | Qwen3.5 4B (base) | Qwen3 4B 2507 | Gemma 4 E4B |
|---|
| MMLU | 73.0 | 70.6 | 64.6 | 65.0 |
| MMLU-Redux 2.0 | 75.4 | 75.4 | 72.4 | 68.8 |
| MMLU-Pro | 68.7 | 71.3 | 66.3 | 66.0 |
| GPQA Diamond | 39.9 | 37.4 | 29.3 | 40.4 |
| SuperGPQA | 35.7 | 33.0 | 29.7 | 32.3 |
| ARC-Challenge | 91.6 | 92.4 | 92.0 | 89.2 |
| HellaSwag | 86.0 | 86.0 | 84.6 | 80.6 |
| Winogrande ✅ | 66.2 | 63.6 | 60.4 | 64.6 |
| TruthfulQA (MC1) | 71.0 | 73.0 | 67.0 | 66.2 |
| BIG-Bench Hard | 76.0 | 75.3 | 67.2 | 78.3 |
| MuSR | 52.1 | 49.6 | 37.1 | 61.7 |
| DROP (F1) ❌ | 60.5 | 63.8 | 34.5 | 37.6 |
| GSM8K | 85.6 | 86.6 | 88.8 | 86.4 |
| MATH-500 | 79.4 | 80.2 | 77.2 | 77.2 |
| IFEval (strict) | 78.8 | 82.3 | 83.6 | 84.7 |
| BFCL tool calling ✅ | 88.7 | 85.2 | 91.7 | 89.9 |
| HumanEval+ | 77.4 | 75.6 | 84.8 | 85.4 |
| MBPP+ | 66.9 | 64.8 | 63.0 | 67.5 |
| Belebele Serbian | 86.6 | 84.6 | 84.2 | 86.8 |
| Belebele Croatian | 85.0 | 84.0 | 83.2 | 87.0 |
| Global-MMLU Serbian | 60.0 | 58.4 | 50.0 | 51.8 |
| MMLU-ProX Serbian | 55.0 | 58.3 | 52.0 | 56.3 |

Thinking mode
Thinking on, up to 32k tokens, sampling 0.6 / 0.95 / 20. Comparisons with other models in thinking mode are still running.
Table with columns: Benchmark, Thinking off, Thinking on, Finished within 32k| Benchmark | Thinking off | Thinking on | Finished within 32k |
|---|
| GSM8K | 85.6 | 85.0 | 100% |
| MATH-500 | 79.4 | 95.0 | 97% |
| MMLU-Pro | 68.7 | 81.0 | 98% |
| GPQA Diamond | 39.9 | 62.6 | 88% |
| SuperGPQA | 35.7 | 47.0 | 92% |
| BIG-Bench Hard | 76.0 | 86.7 | 100% |
| AIME 2024 + 2025 | — | 51.7 | 72% |
Serbian LLM Eval (YugoGPT's suite)
Log-likelihood, 7 tasks × 500 questions, same script and settings for every model (4-bit).
Table with columns: Model, Mean| Model | Mean |
|---|
| Quata2 4B v0.2 | 59.0 |
| YugoGPT 7B | 60.5 |
| Qwen3.5 4B (base) | 57.9 |
| Qwen3 4B 2507 | 51.0 |
Quata's own checks
Targeted tests with the website's system prompt. Small sets (12–300 items), so read them as directional.
Table with columns: Check, Quata2 4B v0.2, Qwen3.5 4B (base), Items| Check | Quata2 4B v0.2 | Qwen3.5 4B (base) | Items |
|---|
| Knows it is Quata2 (no system prompt) | 100.0 | 0.0 | 300 |
| Stays in a customer persona | 100.0 | 97.0 | 100 |
| Facts about Serbia | 75.0 | 62.5 | 12 |
| Knows where companies are | 91.7 | 91.7 | 12 |
| Doesn't put every company in Belgrade | 90.9 | 100.0 | 12 |
| Holds a correct answer under pushback | 75.0 | 65.0 | 60 |
| Fixes a real mistake when challenged | 80.0 | 95.0 | 60 |
| Real tool-using agent tasks, solved | 85.4 | 80.1 | 151 |
| Multi-step agent tasks, solved | 78.4 | 74.5 | 151 |
Reading the numbers: what is noise
Every benchmark is a sample of questions, so every score has a margin of error. For two models on the same
500 questions, the gap needs to be about 2–4 points before it means anything; on the small targeted checks
(20–40 questions) it needs to be 15–25 points. We call a difference real only when it is larger than
twice its standard error, computed pair by pair on the same questions. Anything smaller is a tie, even if
one number is higher: a 1-point gap on a 500-question test is about 5 questions, a coin flip rather than a result.
Get started
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration model_id = "M1n1A1/MiniAI-Quata2-4b-v0.2"tok = AutoTokenizer.from_pretrained(model_id)model = Qwen3_5ForConditionalGeneration.from_pretrained( model_id, torch_dtype="auto", device_map="auto") messages = [{"role": "user", "content": "Ko te je napravio?"}]prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False) # True = think step by step firstinputs = tok(prompt, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=512)print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
The safetensors include the Qwen3.5 vision tower; the GGUF is text-only.
Thinking mode: enable_thinking=True, or put /think / /no_think in a message. Give it room when it thinks:
our thinking-mode numbers use up to 32,000 new tokens.
Run it with llama.cpp
llama-server -m MiniAI-Quata2-4b-v0.2-Q4_K_M.gguf --jinja -c 8192
Recommended sampling: temperature 0.7, top_p 0.8, top_k 20 (thinking: 0.6, 0.95, 20).
Details
- Architecture: Qwen3.5-4B, 4.2B language parameters (+0.33B vision, +0.12B MTP), 32 layers, hidden size 2560, hybrid attention
- Weights: 0.8 × fine-tuned Quata2 + 0.2 × Qwen3.5-4B (WiSE-FT blend)
- Context length: 262,144 tokens in the model config; training used up to 2,048-token sequences
- Precision: bf16 safetensors (~9.7 GB with MTP); Q4_K_M GGUF (~2.8 GB)
- Languages: English first; Serbian, Croatian and Bosnian improved, grammar still behind English
- License: Apache 2.0 (same as the base model)
Honest caveats
- Real losses against the base: DROP. The base is also slightly ahead on IFEval and MMLU-Pro (within noise).
- Qwen3 4B 2507 and Gemma 4 E4B are ahead on coding (HumanEval+) and instruction following (IFEval); Qwen3 4B 2507
also on tool calling (BFCL), Gemma 4 E4B on MuSR, BIG-Bench Hard and Belebele.
- Serbian is improved but not native-level; on YugoGPT's suite YugoGPT is stronger at sentence completion
(HellaSwag, PIQA), Quata at reading and reasoning (BoolQ, ARC).
- Thinking-mode comparisons with other models are still running. Check important answers.
Citation
@misc{miniai2026quata2v02, title = {MiniAI Quata2 4B v0.2}, author = {{MiniAI}}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/M1n1A1/MiniAI-Quata2-4b-v0.2}}, note = {Fine-tuned from and blended with Qwen3.5-4B}}