What it does on real photos
These are transcripts from actual phone-photo runs (hand-picked demos —
measured numbers are in the next section and come only from the
benchmark):
Table with columns: input, ground truth, Warraq output| input | ground truth | Warraq output | |
|---|
| Cheque legal amount, photographed crop (read whole-image) | «أربعة آلاف وستمائة وخمسة عشر» | «أربعة آلاف وستمائة وخمسة عشر .» | ✅ exact |
| School sentences on ruled paper, phone photo — line 1 of 3 | «سوف يزرع الفلاح البطاطس في الخريف .» | «- سوف يزرع الفلاح البطاطس في الخريف .» | ✅ exact |
| 12-line poem on ruled notebook paper, phone photo — one of 12 auto-split lines | «سهلنا والجبل منبت للرجال» | «سهلنا والجبل منبت للرّجال» | ✅ exact |
| Same school photo — line 3 (one letter missed, shown for honesty) | «لا تحاولوا أن تتسلقوا هذه الشجرة العالية .» | «- لا تحاولوا أن تسلقوا هذه الشجرة العالية .» | ⚠️ one letter |
It also ships with the practical machinery real documents need: an
automatic line splitter for multi-line pages (photo-robust: rulings,
cross-line descenders, stanza marks) and grammar-constrained decoding
for cheque amounts (snaps a noisy reading to the nearest valid Arabic
amount phrase).
On the held-out KHATT v1.0 test split (1038 real
handwritten lines), verbatim scoring (diacritics counted), greedy
decoding:
Table with columns: model, CER ↓, WER ↓, exact lines| model | CER ↓ | WER ↓ | exact lines |
|---|
| Warraq (this adapter) | 3.86% | 16.55% | 29.29% |
| Arabic-English-handwritten-OCR-v3 (best other model) | 5.86% | 18.30% | 28.81% |
| Fanar-2-Oryx-IVU (same base, zero-shot) | 50.98% | 82.58% | 0.10% |

Fair-comparison note, read it: Warraq's training data includes the
KHATT training split, so its KHATT-test score is an in-domain result.
The other models never saw KHATT — they are measured zero-shot under
the same prompt, decoding and normalization. The table answers "who
reads this handwriting best today", not "whose base model is smartest".
Full protocol, all 9 models, distribution stats and charts:
evaluation report on GitHub · raw numbers:
benchmark.json.
Quickstart
from transformers import AutoModelForImageTextToText, AutoProcessorfrom peft import PeftModelfrom PIL import Image BASE = "QCRI/Fanar-2-Oryx-IVU"processor = AutoProcessor.from_pretrained(BASE)model = AutoModelForImageTextToText.from_pretrained(BASE, device_map="auto")model = PeftModel.from_pretrained(model, "mabdulaziz499/Warraq-Arabic-HTR-7B") image = Image.open("line.jpg") # one handwritten LINE works bestmessages = [{"role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": "اكتب النص المكتوب بخط اليد في هذه الصورة كما هو تماما."},]}]inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=128, do_sample=False)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
For multi-line pages, split into lines first (any line segmenter works);
for cheque amounts, constrain the output to the Arabic amount grammar —
both behaviors are what the demos above used.
Table | |
|---|
| Method | QLoRA (4-bit NF4 base, bf16 compute), LoRA r=16 α=32, attention+MLP |
| Trainable | ≈40M parameters (0.48% of Fanar-2-Oryx-IVU) |
| Data | 24,487 lines: KHATT v1.0 + OnlineKhatt + Muharaf + hard-degraded synthetic (gray paper, watermarks, deep low-res, JPEG — cheque-like conditions) |
| Steps | 4000 (≈2.6 epochs), cosine schedule |
| Leakage guard | every training line whose exact text appears in the eval set is dropped |
| Text order | all sources auto-checked for visual-order (reversed) ground truth and normalized to logical order |
Honest limitations
- Ornamental / fully-vocalized calligraphy (Quranic naskh, decorative
scripts) is a different domain — Warraq targets everyday handwriting
and will under-read calligraphy.
- Historical manuscripts improve over the base but remain hard.
- Very low-resolution crops of fused number-words can garble; use the
whole crop plus amount-grammar snapping, as banks do.
- Trained and evaluated on line-level images; page reading relies on a
line splitter.
License & credits
Adapter: Apache-2.0 (matching the base model). Base:
QCRI/Fanar-2-Oryx-IVU by QCRI. Training data
credits: KHATT v1.0 (research terms), OnlineKhatt, Muharaf. If you use
Warraq, please also cite KHATT (Mahmoud et al., Pattern Recognition 2014)
and the base model.