Why this adapter and not the stronger word-OCR variant
We trained two arms on the same data and evaluated them under one preregistered selection rule:
- M1 (this adapter): LoRA on decoder layers only. Vision tower and vision-language merger frozen.
- M2: LoRA on decoder and vision tower, plus a trainable merger.
M2 achieved better word-recognition accuracy (Dev CER 5.98% vs 7.78%; paired difference โ1.80 pp, 95% CI โ2.43 to โ1.14) but failed the preregistered table-structure preservation gate (exact table-structure match 0.80 vs 0.98 for the base model on 50 sealed synthetic table pages; tolerance 0.04). M1 was selected to preserve document-structure performance on the evaluated benchmarks.
The M2 checkpoint is kept as a separate research model and is not part of this repository.
Usage
The adapter must be used with the exact protocol below. Other prompts, image sizes or sampling settings were not evaluated.
Table with columns: Setting, Value| Setting | Value |
|---|
| Base model | ATH-MaaS/OvisOCR3 at revision 4833d5d2afa6e769ee933f024504d67ace80a24c (the base repo's main has moved since; pin this revision) |
| Prompt | prompts/plain_ocr.txt from the base repo at that revision (sha256 e037cf00โฆ) |
| Chat template | apply_chat_template(..., add_generation_prompt=True, enable_thinking=False); image first, then the prompt |
| Image size | min_pixels = 448*448, max_pixels = 2880*2880 |
| Decoding | greedy (do_sample=False), max_new_tokens=64 |
| Weights | base in fp32, generation under bf16 autocast (as evaluated) |
| Input | one RGB crop containing one word |
pip install "transformers>=5.0" "peft>=0.21" torch pillow huggingface_hubpython inference_example.py --image word.pngpython inference_example.py --image word.png --report # JSON with load checks
Minimal code (same as inference_example.py):
import torch, transformersfrom huggingface_hub import hf_hub_downloadfrom peft import PeftModelfrom PIL import Imageย BASE, REV = "ATH-MaaS/OvisOCR3", "4833d5d2afa6e769ee933f024504d67ace80a24c"processor = transformers.AutoProcessor.from_pretrained(BASE, revision=REV)model = transformers.AutoModelForImageTextToText.from_pretrained(BASE, revision=REV, dtype=torch.float32).to("cuda")model = PeftModel.from_pretrained(model, "ThakiCloud/ThakiOCR-KO-v1").eval()ย prompt_text = open(hf_hub_download(BASE, "prompts/plain_ocr.txt", revision=REV), encoding="utf-8").read()msgs = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt_text}]}]prompt = processor.tokenizer.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = processor(text=[prompt], images=[Image.open("word.png").convert("RGB")], return_tensors="pt", images_kwargs={"min_pixels": 448 * 448, "max_pixels": 2880 * 2880}).to("cuda")with torch.no_grad(), torch.autocast("cuda", dtype=torch.bfloat16): out = model.generate(**inputs, max_new_tokens=64, do_sample=False)print(processor.tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())
Measured on one B200 GPU at batch size 1 over the 2,400 Dev crops: median latency 0.210 s, p95 0.360 s, peak memory 7.02 GiB (base model alone: 0.186 s / 0.310 s / 4.95 GiB).
Results
Word OCR (Dev, 2,400 crops, 600 per source)
CER after NFC normalization and whitespace stripping. Lower is better.
Table with columns: Source, Base (OvisOCR3), M1 (this adapter), M2 (not released here)| Source | Base (OvisOCR3) | M1 (this adapter) | M2 (not released here) |
|---|
| Pooled | 25.28% | 7.78% | 5.98% |
| 605 handwriting | 28.32% | 7.23% | 5.53% |
| 91 handwriting (15 writers) | 66.89% | 17.38% | 12.19% |
| 91 printed | 6.28% | 1.56% | 1.66% |
| 105 outdoor photos | 13.23% | 7.97% | 6.47% |
- M1 vs base, paired over the same items: pooled โ17.50 pp (95% CI โ20.32 to โ15.12). Every source improves with a CI excluding zero.
- M2 vs M1: pooled โ1.80 pp (95% CI โ2.43 to โ1.14); 91 handwriting โ5.19 pp with a writer-clustered bootstrap over 15 writers (95% CI โ7.49 to โ3.20). M2 is the better word recognizer.
- By character class (M1): Hangul 9.11%, digits 0.85%, Latin 5.04%, punctuation 4.14% (base: 18.87%, 3.75%, 22.30%, 18.42%). Latin covers only 42 crops (139 characters).
Document structure (no-regression check against the base model)
These pages are not word crops. The check asks only whether fine-tuning broke abilities the base model already had. The numbers were measured on a bf16 merged copy of this adapter served under the same interface as the base model.
Table with columns: Check, Tolerance, Base, M1, M2| Check | Tolerance | Base | M1 | M2 |
|---|
| Exact table-structure match (50 sealed synthetic table pages) | โ0.04 | 0.98 | 1.00 | 0.80 โ |
| Table TEDS (sealed) | โ0.02 | 0.999 | 0.992 | 0.992 |
| Reading-order Kendall ฯ (sealed) | โ0.01 | 0.998 | 0.997 | 0.995 |
| Reading-order coverage (sealed) | โ0.01 | 0.998 | 0.997 | 0.996 |
| Korean document test suite (849 items), table pass rate | โ0.02 | 51.9% | 56.2% | 55.6% |
| Korean document test suite, overall pass rate | โ0.02 | 58.0% | 60.2% | 59.5% |
M1's 1.00 is a result on 50 synthetic sealed pages. It does not mean table structure is always correct on real documents.
What we measured but do not claim
- No full-page OCR claim. Full-page reading of real documents was not evaluated as a target task.
- No comparison with larger models. We do not claim this 0.9B-class adapter matches larger OCR models.
- No out-of-distribution claim. Dev crops come from the official validation splits of the same three datasets used for training. They are not an independent test set. For 91 printed, the same fonts appear in train and dev.
Training
- Base:
ATH-MaaS/OvisOCR3 @ 4833d5d2, original weights (no earlier fine-tune as starting point).
- LoRA r=16, alpha=32, dropout 0.05, on
q/k/v/o/gate/up/down_proj of the language model only (96 modules, 6.39M trainable parameters). Vision tower and merger frozen.
- Early stopping on an internal validation split carved from the training data (patience 4, min delta 0.002). Training stopped at step 5,500; the released adapter is the best checkpoint at step 5,250. One B200 GPU, 4.74 GPU-hours, peak training memory 3.88 GiB.
- The Dev set was used only for evaluation, never for checkpoint selection.
Training data
This adapter was trained on data from AI ํ๋ธ (AI-Hub, Korea's public AI training-data platform):
- AI ํ๋ธ(AI-Hub) โ ๋์ฉ๋ ์๊ธ์จ OCR ๋ฐ์ดํฐ (605) โ large-scale handwriting OCR data
- AI ํ๋ธ(AI-Hub) โ ๋ค์ํ ํํ์ ํ๊ธ ๋ฌธ์ ์ด๋ฏธ์ง ์ธ์ ๋ฐ์ดํฐ (91) โ Korean character image recognition data in various forms (handwritten and printed subsets)
- AI ํ๋ธ(AI-Hub) โ ์ผ์ธ ์ค์ ์ดฌ์ ํ๊ธ ์ด๋ฏธ์ง ๋ฐ์ดํฐ (105) โ Korean text images photographed outdoors
Data redistribution is not permitted. The training and evaluation data, and anything derived from it (crops, label files, subsets, samples), are not included in this repository and are not shared. The data was used only for internal training and evaluation by the approved applicant. This card contains no sample images or texts from these datasets. To use the data, apply for it yourself on AI-Hub.
The printed subset of dataset 91 was rendered with third-party fonts whose rights are separate from AI-Hub's approval. We checked the font-license field of every label used for training (10 fonts: 7 under the SIL Open Font License, 3 free fonts published by local governments) and found no prohibition. This repository contains no font files and no rendered font images; only the LoRA weights are released.
Files
Table with columns: File, Purpose| File | Purpose |
|---|
adapter_model.safetensors | LoRA weights (25.6 MB), sha256 28b56eb5e998437536c641be01d24a7b582fcf01f8796c53c61e871c2f38193e |
adapter_config.json | PEFT config pointing to ATH-MaaS/OvisOCR3 at the pinned revision |
inference_example.py | Reference inference with the evaluated protocol and load checks |
MODEL_CARD.ko.md | Korean version of this card |
LICENSE | Apache License 2.0 |
A bf16 merged checkpoint (1.71 GB) exists internally and was used for the document-structure checks. It is not part of this release. A bf16 merge is not bit-identical to running the fp32 base with this adapter.
Blog posts
Write-ups of this model on the ThakiCloud tech blog:
License
Apache License 2.0, the same license as the base model (see LICENSE). Use of the base model is governed by its own license. When you use this adapter, keep the attribution "AI ํ๋ธ(AI-Hub)" and the dataset names above.