aghasalim

siba-meristem-2.0-preview

README

License: apache-2.0

Results

azbench: 2,148 zero-shot multiple-choice questions in Azerbaijani (Belebele reading, INCLUDE exams, TUMLU school), exact-letter scoring, temperature 0, thinking off.

Full precision (transformers, bf16, same harness and GPU for both):

Table with columns: Reading, Exams, School, Average
	Reading	Exams	School	Average
Base model	80.4 %	63.1 %	59.4 %	67.7 %
Meristem 2.0 preview	80.4 %	65.1 %	63.4 %	69.7 %

Per question against the base: school 74 newly right, 46 newly wrong (McNemar p = 0.014); exams 46 and 35 (p = 0.27, not significant); reading 27 and 27.

Ollama, Q4_K_M (the GGUF in this repo, the same setup as the public azbench leaderboard: NVIDIA L4, Ollama, thinking off; both models run on the same machine):

Table with columns: Reading, Exams, School, Average
	Reading	Exams	School	Average
Base model	79.9 %	58.9 %	55.6 %	64.8 %
Meristem 2.0 preview	78.7 %	61.5 %	63.3 %	67.8 %

Per question against the base: school 96 newly right, 42 newly wrong (McNemar p < 0.0001); exams 62 and 48 (p = 0.22); reading 36 and 47 (p = 0.27). The base model reproduced its leaderboard score exactly on this run.

What it is

Method: LoRA (rank 32, alpha 64) on the language model's attention and MLP projections; the vision tower is untouched. Supervised fine-tuning for 2 epochs, loss on the answer letter only.
Training data: 6,179 Azerbaijani multiple-choice questions (3,753 reading, 2,426 standalone knowledge) that the base model wrote from 8,000 Azerbaijani passages. A question was kept only if the same model, shown its passage, picked the keyed answer; duplicates were removed and answer letters balanced so A, B, C and D are each right a quarter of the time.
Passages: Azerbaijani Wikipedia (CC BY-SA 4.0), Azerbaijani Wikisource (CC BY-SA 3.0 / GFDL), laws, decrees and orders from president.az and meclis.gov.az (official documents, not subject to copyright under Azerbaijan's Law on Copyright and Related Rights, art. 7), and FinePDFs Azerbaijani (ODC-By 1.0).
Decontamination: every passage and every generated question was checked against all 2,148 azbench items; any shared 13-word sequence was dropped. None of the benchmark's questions or passages were used.

Files

Table with columns: File, What
File	What
`SIBA-Meristem-2.0-preview.Q4_K_M.gguf`	The merged model, 4-bit, text only, for Ollama, llama.cpp and LM Studio
`Modelfile`	Ollama Modelfile (chat template and settings)
`adapter_model.safetensors`, `adapter_config.json`	The LoRA adapter, for transformers + PEFT on top of the base model (below)
`LICENSE`	Apache License 2.0

Use

bash

ollama run aghasalim/siba-meristem-2.0-preview

python

import torchfrom transformers import AutoProcessor, AutoModelForImageTextToTextfrom peft import PeftModel proc = AutoProcessor.from_pretrained("Qwen/Qwen3.5-9B")model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-9B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(model, "aghasalim/siba-meristem-2.0-preview")

Limits

Part of the gain is test-taking. It was trained on multiple-choice questions in the same layout azbench uses. The school gain is the largest and the only statistically solid one; reading did not improve at all.
The training questions are synthetic and of mixed quality: some keyed answers repeat words from the question, and some law questions are trivia (decree numbers and dates).
Open-ended facts can be wrong. It was trained to pick answers, not to write them. Asked in free text when Azerbaijan's Constitution was adopted, it answered 1993 and 1998 (it was 1995). Check facts; use it for multiple-choice, reading and drafting rather than as a source of truth.
Not separately safety-tuned. It inherits the base model's behaviour; nothing was added.
Text only in the GGUF. The adapter leaves the base model's vision tower unchanged.

Technical details and licence

Fine-tuned by SIBA from Qwen3.5-9B (Apache 2.0, Qwen team, Alibaba Cloud). Apache License 2.0, the same as the base model; modified by SIBA ("Süni İntellekt Biznes Avtomatlaşdırma" MMC) with the LoRA adapter described above.

Earlier Meristem 1.x releases are separate, older models under a research-only licence: see aghasalim/siba-meristem-1.0.

Contact

Aghasalim Mustafazada, SIBA · siba.az · info@siba.az

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

aghasalim

Model Tree

Base

Qwen/Qwen3.5-9B

Adapter

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality