Results
These are the 480 test pages of the dataset. They come from whole books never seen in training: Buddha Jayanti Tripitaka volumes 12, 29 and 47, two pitaka.lk books, held-out acts pages and library pages. CER is the character error rate after normalisation, with bold markup removed. The other columns are F1 scores.
Table with columns: source, pages, CER, median page CER, bold F1, line-break F1, paragraph F1| source | pages | CER | median page CER | bold F1 | line-break F1 | paragraph F1 |
|---|
| pitaka-books (old prints) | 200 | 1.95% | 0.93% | 0.63 | 0.92 | 0.87 |
| acts (1981–89) | 82 | 1.49% | 0.88% | n/a | 0.92 | 0.82 |
| bjt (Buddha Jayanti Tripitaka) | 186 | 2.11% | 1.46% | 0.17 | 0.86 | 0.91 |
| opus-lib (library books) | 12 | 5.4% | 5.6% | 0.62 | 0.66 | 0.33 |
| all | 480 | 2.0% | 1.1% | 0.56 | 0.89 | 0.86 |
The base LightOnOCR-2-1B scores 160% CER on the same pages: it does not read Sinhala.
The reference texts come from corrected editions, which differ from the printed page in places, so about 1–2% of the measured error is not the model's. The BJT bold score is low, possibly because the BJT reference text marks bold differently from the print; this has not been checked against the scans. The opus-lib references are machine transcriptions.
Usage
Pages are read at up to 1540 px on the longest edge, the resolution used in training. Use greedy decoding. Dense pages can need more than 4,096 output tokens, because Sinhala costs about one token per character.
vLLM (recommended for many pages)
from vllm import LLM, SamplingParamsfrom PIL import Image llm = LLM(model="pathnirvana/sinhala-pali-ocr-lightonocr-1b", max_model_len=12288, limit_mm_per_prompt={"image": 1})sp = SamplingParams(temperature=0, max_tokens=6144, repetition_penalty=1.05)img = Image.open("page.jpg").convert("RGB")out = llm.chat([[{"role": "user", "content": [{"type": "image_pil", "image_pil": img}]}]], sp)print(out[0].outputs[0].text)
On one RTX 5090 with vLLM 0.32, this ran the 480 test pages at about 1.45 pages per second.
import torchfrom PIL import Imagefrom transformers import LightOnOcrForConditionalGeneration, LightOnOcrProcessor repo = "pathnirvana/sinhala-pali-ocr-lightonocr-1b"model = LightOnOcrForConditionalGeneration.from_pretrained(repo, dtype=torch.bfloat16).to("cuda")proc = LightOnOcrProcessor.from_pretrained(repo)conv = [{"role": "user", "content": [{"type": "image", "image": Image.open("page.jpg").convert("RGB")}]}]inputs = proc.apply_chat_template(conv, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to("cuda", torch.bfloat16)out = model.generate(**inputs, max_new_tokens=6144, do_sample=False, repetition_penalty=1.05)print(proc.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Transformers 5 may warn about the tokenizer's regex pattern. That makes no difference for this model: tokens matched the training tokenizer on every test text checked.
Apple Silicon (mlx-vlm)
mlx-vlm 0.7.12 runs this model on the Mac GPU with the same output as vLLM. On an M4 that is about 50 tokens/s, or roughly 50 seconds per page.
pip install mlx-vlm==0.7.12python -m mlx_vlm.generate --model pathnirvana/sinhala-pali-ocr-lightonocr-1b --image page.jpg --prompt "" --max-tokens 6144 --temperature 0
Pass --prompt "": the CLI's default prompt is a question the model was not trained on.
Prompt
The model is trained to work from the image alone, with no text prompt. One text instruction was used in training:
Table with columns: prompt, effect| prompt | effect |
|---|
| none | full format, as above |
nobold | the same, without bold markup |
Training
-
Data: 7,734 training pages from the dataset:
- 2,973 Buddha Jayanti Tripitaka pages (Pali and Sinhala)
- 4,395 pages of old prints aligned with corrected editions from pitaka.lk
- 289 pages of Sri Lankan acts (1981–89)
- 77 library pages transcribed by Claude Opus
-
Method: full fine-tune of all weights, including the vision encoder.
- Precision: fp32 master weights with bf16 autocast, gradient checkpointing.
- Loss on the target text only.
- Learning rate 2e-5, cosine schedule, 3% warmup.
- Effective batch of 16 pages, 2 epochs (964 steps).
- Input resolution 1540 px longest edge, randomly reduced to 82–100% in training.
- Mild scan-like augmentation: small rotation, contrast, blur, JPEG.
-
Hardware: one RTX 5090 (32 GB), about 8 hours.
-
Validation loss:
Table with columns: step, 200, 400, 600, 800, 964| step | 200 | 400 | 600 | 800 | 964 |
|---|
| loss | 0.0505 | 0.0405 | 0.0362 | 0.0348 | 0.0345 |
Limitations
- Coverage: trained on Sinhala-script Sinhala and Pali book pages. Other scripts, tables, and handwriting are untested.
- Typewritten and very damaged pages are rare in the training data.
- Not word-perfect: expect about 2% character errors on typical old prints, more on stained or low-resolution scans.
- Bold is less reliable than the text.
Credits