GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
wjbmattingly
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
wjbmattingly
Model Tree
lightonai/LightOnOCR-3-0.8B
Input Modalities
Output Modalities
Supported Functionality
243 held-out pages, each in a different hand, none seen in training. CER is quoted after NFD normalisation, because the corpus writes ũ both precomposed and as u + combining tilde, and charging two edits for one glyph measures the encoding rather than the reading.
| model | CER (NFD) | macro | WER (NFD) | line recall (NFD) | truncated |
|---|---|---|---|---|---|
| LightOnOCR-3-0.8B, zero-shot (same job) | 0.4654 | 0.4557 | 0.9265 | 0.0024 | 7.8% |
| this model | 0.1206 | 0.1018 | 0.3613 | 0.1678 | 0.4% |
| Qwen3.5-0.8B, full FT — exact control | 0.1458 | 0.1318 | 0.4029 | 0.1672 | 1.6% |
| LightOnOCR-3-4B, full FT | 0.1228 | 0.1036 | 0.3605 | 0.1681 | 0.4% |
| chandra-ocr-2 (5.3B), LoRA | 0.1232 | 0.1039 | 0.3639 | 0.1668 | 0.8% |
| Qwen3.5-2B, full FT | 0.1251 | 0.1066 | 0.3616 | 0.1647 | 0.8% |
| Qwen3.5-9B, full FT | 0.1168 | 0.0985 | 0.3541 | 0.1751 | 0.4% |
Greedy decoding, 3,072 max new tokens, 2,048 visual tokens per page, eval batch 4. Treat differences under 0.02 CER as ties — four seeds of one configuration in this project span 0.021.
OCR pretraining is worth 0.025 CER here, measured against an exact control.
Qwen3.5-0.8B fine-tuned with the identical recipe — same 9,906 pages, learning
rate, epochs, seed, effective batch, image budget, holdout, and evaluation GPU type —
scores 0.1458. The only difference is the starting checkpoint. The gap is outside
this project's noise floor (four seeds of one configuration span 0.021), so it is a
result rather than a fluctuation.
That puts a 0.8B level with models 2.5 to 6.6 times its size: chandra-ocr-2 (0.1232), the 2B (0.1251), and the LightOnOCR-3-4B fine-tune (0.1228) — all within 0.02 of it.
This model and the comparison rows were scored on RTX PRO 6000 Blackwell GPUs, except the LightOnOCR-3-4B and Qwen3.5-9B rows, which were scored on H200s. That matters: the same untrained LightOnOCR-3-4B scored 0.0202 CER apart on those two GPU types (bf16 arithmetic differs and greedy decoding amplifies it), so gaps under ~0.03 between this model and the H200 rows are ties. The comparisons with the zero-shot row and the RTX-scored rows are like-for-like.
python
import torchfrom PIL import Imagefrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForImageTextToText, AutoProcessor REPO = "wjbmattingly/comma-lightonocr-3-0.8b"model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(REPO) # The prompt is part of the model: it was trained to answer this exact instruction.# Use it verbatim -- LightOnOCR's own empty transcription prompt is a different task.prompt = open(hf_hub_download(REPO, "prompt.txt"), encoding="utf-8").read() image = Image.open("page.jpg").convert("RGB")messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3072, do_sample=False, repetition_penalty=1.1, use_cache=True)print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))prompt.txt is required. This model was trained on a 11,629-character instruction,
shipped in this repo; LightOnOCR-3 itself was trained on an empty prompt and emits
block-labelled markdown with bounding boxes, which is not this task.enable_thinking=False. Passed explicitly so the generation prompt closes the
thinking block regardless of which template variant is loaded.processor_config.json is capped at 2,048 visual
tokens, the budget used in training and scoring. LightOnOCR-3 ships uncapped
(~16k tokens); loading a stock processor shows the page at a resolution this model
never trained on.repetition_penalty=1.1 is this project's measured serving setting: it removes most
looping on sparse pages at a small cost in line recall. The table above is greedy
(penalty 1.0); set 1.0 to reproduce it.generation_config.json sets use_cache: true. LightOnOCR-3-0.8B ships false (a
training artefact), which makes generation recompute the whole sequence at every
step; every number above was produced with the cache on.Full fine-tune of all 852,985,920 parameters (vision tower included), 3 epochs, 3,717 steps, learning rate 5e-05, cosine schedule, effective batch 8, bf16, 2,048 visual tokens per page, on RTX PRO 6000 Blackwell. 2 of 9,906 pages exceeded the 8,192-token length budget and were dropped (the stock-Qwen control dropped 1).
Data: comma-project/deep-jsonl.
A research artifact for cultural-heritage and digital-humanities work, not a production OCR service. Trained on one collection, it should be expected to degrade on other hands, languages, genres and centuries. It reproduces what is written, including historical spelling and language; transcribing a record is not an endorsement of its contents. Verify any name, date or figure before relying on it.
import torchfrom PIL import Imagefrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForImageTextToText, AutoProcessor REPO = "wjbmattingly/comma-lightonocr-3-0.8b"model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(REPO) # The prompt is part of the model: it was trained to answer this exact instruction.# Use it verbatim -- LightOnOCR's own empty transcription prompt is a different task.prompt = open(hf_hub_download(REPO, "prompt.txt"), encoding="utf-8").read() image = Image.open("page.jpg").convert("RGB")messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3072, do_sample=False, repetition_penalty=1.1, use_cache=True)print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))