amad-iq

amad-vlm5

README

License: apache-2.0

Results on KITAB-Bench

KITAB-Bench ocr-eval, all 13 datasets, 3,760 images, scored on the final transcription only with the benchmark's unmodified metrics and Arabic normalization.

Table with columns: Model, CHrF ↑, CER ↓, WER ↓
Model	CHrF ↑	CER ↓	WER ↓
amad-vlm5	81.05	0.25	0.36
AIN-7B	78.33	0.20	0.28
Gemini-2.0-Flash	77.95	0.13	0.32
GPT-4o	61.01	0.31	0.55
Qwen2.5VL-7B	49.23	1.20	1.41
GPT-4o-mini	47.21	0.43	0.71
EasyOCR	45.47	0.58	0.89
Tesseract	39.62	0.54	0.84
Qwen2VL-7B	33.94	1.48	1.55
Surya	20.61	4.95	5.61
Paddle	16.73	0.79	1.02

Baselines are the published KITAB-Bench numbers. Read these two caveats before quoting a rank:

Training overlap. 552 benchmark images (khatt 200/200, onlinekhatt 181/200, muharaf 171/200) also occur in the model's training data. Excluding those three datasets entirely, the score is CHrF 77.35 / CER 0.32 / WER 0.43 over the remaining 3,160 images.
Two outliers dominate CER. Two of the 3,760 outputs degenerate into a repeated phrase; without them CER is 0.13 and WER 0.28. CHrF, which is bounded, is the more stable summary.

Per-dataset results (final-answer scoring, 4,096 tokens):

Table with columns: Dataset, Samples, CER ↓, WER ↓, CHrF ↑
Dataset	Samples	CER ↓	WER ↓	CHrF ↑
patsocr	500	0.01	0.06	96.28
onlinekhatt	200	0.02	0.08	95.75
khatt	200	0.03	0.16	93.83
synthesizear

Methodology note. amad-vlm5 is a thinking VLM. For OCR scoring, only the final transcription is evaluated; reasoning text is removed before metric calculation. The benchmark was run with greedy decoding in a 4-bit-quantized inference configuration; the bf16 and GGUF files in this release were not separately re-benchmarked, so small differences from the table are expected.

Files

Table with columns: Repository, Contents, Size, Use
Repository	Contents	Size	Use
`amad-iq/amad-vlm5` (this repo)	bf16 safetensors	16.60 GB	Transformers, vLLM, further fine-tuning
`amad-iq/amad-vlm5-GGUF`	`amad-vlm5-f16.gguf`	15.24 GB	llama.cpp / LM Studio, full precision

The GGUF language model files do not work without the mmproj file — it carries the vision encoder. Download it into the same folder as the model file.

Quick start

Transformers

python
import re
import torch
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

repo = "amad-iq/amad-vlm5"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    repo, dtype=torch.bfloat16, device_map="auto"
)
processor = AutoProcessor.from_pretrained(repo)

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "page.png"},
        {"type": "text", "text": "Extract the text in the image. Give me the final text, nothing else."},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    out = model.generate(
        **inputs, max_new_tokens=4096, do_sample=False, repetition_penalty=1.05
    )
raw = processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]
text = re.sub(r"<think>.*?</think>", "", raw, flags=re.S).strip()
print(text)

llama.cpp

bash
llama-mtmd-cli \
  -m amad-vlm5-q8_0.gguf \
  --mmproj mmproj-amad-vlm5-f16.gguf \
  --image page.png \
  -p "Extract the text in the image. Give me the final text, nothing else." \
  -n 4096 --temp 0 --repeat-penalty 1.05

LM Studio

Search for amad-iq/amad-vlm5-GGUF in LM Studio, download a model file together with mmproj-amad-vlm5-f16.gguf, load the model, and attach an image. Set the context length to at least 8192 and the maximum output tokens to 4096 or more.

Handling the thinking block

The model may emit <think>…</think> before the transcription. <think> is ordinary text, not a special token, so it appears in decoded output. Keep only what follows the last </think>:

python
import re
def final_text(raw: str) -> str:
    if "</think>" in raw:
        raw = raw.rsplit("</think>", 1)[1]
    return re.sub(r"^<think>.*", "", raw, flags=re.S).strip()

If the output contains <think> but no </think>, the generation ran out of budget before finishing; raise max_new_tokens and retry.

Intended use and training data

amad-vlm5 is intended for transcribing Arabic-script text from images: books, manuscripts, forms, screenshots, and handwritten notes. It was fine-tuned on a mixture of public Arabic OCR datasets covering printed, handwritten, historical, and synthetic text, including some subsets that overlap with KITAB-Bench (see the caveat above). It is not a general assistant and has not been evaluated for languages other than Arabic and English.

Limitations

Runaway reasoning (≈0.6% of benchmark pages). On some very dense pages the model reasons for the entire budget and never emits a transcription. Detect this by the missing </think> and retry with a larger budget or a different crop.
Repetition loops (rare). Greedy decoding can occasionally lock onto a repeated phrase and run to the token limit. A repetition penalty of 1.05 mitigates this.
Quantized variants are not separately benchmarked. Expect Q4_K_M to be slightly worse than Q8_0 or bf16 on handwritten and historical material.

License

Released under the Apache 2.0 license, the same license as the Qwen2.5-VL-7B-Instruct base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

amad-iq

Model Tree

Base

Qwen/Qwen2.5-VL-7B-Instruct

Fine-tuned

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer