Model Details
- Base model:
Qwen/Qwen2.5-VL-7B-Instruct, revision cc594898137f460bfe9f0759e9844b3ce807cfb5
- Adapter: LoRA, r=16, alpha=32, dropout=0.05
- Target modules:
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Prompt used during training/inference:
"Transcribe the handwritten Thai sentence in the image exactly as written. Answer with the transcription only."
Dataset
Trained on iapp/thai_handwriting_dataset (13,550 rows, single train split; the
training pipeline carves its own train/val/test from it).
Evaluation
Table with columns: Test set, N, CER, char F1, Exact match| Test set | N | CER | char F1 | Exact match |
|---|
| burapha (Thai characters) | 13,600 | 1.2451 | 0.1315 | 14.68% |
| hme (handwritten maths) | 24,607 | 0.3428 | 0.7923 | 15.29% |
| thai_sentence (Thai prose) | 2,032 | 0.1049 | 0.9362 | 38.34% |
Findings
1. Best Thai result in the project, by a wide margin. On Thai sentences it reaches
CER 0.1049 / char F1 0.9362, against e1 (BURAPHA characters) at
0.5041 and e3 (joint) at 0.6753 — roughly 6× lower error than e3.
Sentence-level training is what buys this; character-level training does not transfer to
connected script.
2. The converse is also true: it is bad at isolated characters. On BURAPHA it scores
CER 1.2451 — worse than 1.0, i.e. it emits more erroneous characters than the reference
contains. A sentence model asked for a single glyph produces a sentence-shaped answer. Do
not use this adapter for character-level OCR; use TLDRKKU/burapha.
3. It is not a maths reader (hme CER 0.3428). In the THIRA pipeline Thai-script
regions are routed here and symbolic/mathematical regions to e3; routing beat either
adapter alone on real worksheets (symbol-level CER 0.453 routed, vs 0.532 this adapter
alone and 0.507 e3 alone, n=34 matched regions).
4. It flattens mathematical structure. Across 1,145 real worksheet lines, only 10.0%
of this adapter's readings carry LaTeX structure, against 34.7% for e3; \\frac appears on
38 lines versus 170. That matters downstream because mathematical verification needs the
structure, and character error rate cannot see it. This is the measured reason THIRA routes
maths away from this adapter despite its lower headline CER.
5. Real worksheets are harder than the benchmark. On real Thai student homework the
routed configuration scores symbol-level CER 0.453 (n=34 matched regions), against
0.1049 here. Treat the benchmark number as in-domain only.
How to Use
import torch
from peft import PeftModel
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
base = Qwen2_5_VLForConditionalGeneration.from_pretrained(
"Qwen/Qwen2.5-VL-7B-Instruct",
revision="cc594898137f460bfe9f0759e9844b3ce807cfb5",
torch_dtype=torch.bfloat16,
)
model = PeftModel.from_pretrained(base, "TLDRKKU/thai-sentence")
TLDRKKU/burapha — BURAPHA-only Thai character LoRA (e1)
TLDRKKU/HMk100 — HME100K-only maths LoRA (e2)
TLDRKKU/burapha-HMk100 — joint adapter (e3), the selected maths recogniser
TLDRKKU/thai-sentence — Thai sentence LoRA (e5), the selected Thai recogniser
Part of THIRA — Thai Handwriting Intelligence Recognition & Analysis.