GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
wjbmattingly
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
wjbmattingly
Model Tree
lightonai/LightOnOCR-3-4B
Input Modalities
Output Modalities
Supported Functionality
243 held-out pages, each in a different hand, none seen in training. CER is quoted after NFD normalisation, because the corpus writes ũ both precomposed and as u + combining tilde, and charging two edits for one glyph measures the encoding rather than the reading.
| model | CER (NFD) | macro | WER (NFD) | line recall (NFD) | truncated |
|---|---|---|---|---|---|
| LightOnOCR-3-4B, zero-shot (same job) | 0.3876 | 0.3486 | 0.8218 | 0.0056 | 2.9% |
| this model | 0.1228 | 0.1036 | 0.3605 | 0.1681 | 0.4% |
| LightOnOCR-3-0.8B, full FT | 0.1206 | 0.1018 | 0.3613 | 0.1678 | 0.4% |
| chandra-ocr-2 (5.3B), LoRA | 0.1232 | 0.1039 | 0.3639 | 0.1668 | 0.8% |
| Qwen3.5-2B, full FT | 0.1251 | 0.1066 | 0.3616 | 0.1647 | 0.8% |
| Qwen3.5-9B, full FT | 0.1168 | 0.0985 | 0.3541 | 0.1751 | 0.4% |
Greedy decoding, 3,072 max new tokens, 2,048 visual tokens per page, eval batch 4. Treat differences under 0.02 CER as ties — four seeds of one configuration in this project span 0.021.
There is no exact control for this model, and it is no better than the 0.8B. Both
attempts to fine-tune stock Qwen3.5-4B on these pages hit the cluster's 48-hour limit
before finishing, so the 4B can only be bracketed against neighbours. It lands at
0.1228 against the LightOnOCR-3-0.8B fine-tune's 0.1206 — within noise, and evaluated on
different hardware (see below). Five times the parameters bought nothing measurable
on this task; this project has seen the same plateau around 2B with stock Qwen.
Prefer comma-lightonocr-3-0.8b unless you have reason to want the larger model.
This model was evaluated on an H200; most comparison rows on an RTX PRO 6000. That matters more than it sounds: untrained LightOnOCR-3-4B scored 0.3674 on one GPU type and 0.3876 on the other — the same weights, pages and code, 0.0202 apart. bf16 arithmetic differs between architectures and greedy decoding amplifies it. Treat gaps under ~0.03 between this model and RTX-scored rows as ties. The zero-shot row above was scored in the same job as this model, so that one comparison is exact.
python
import torchfrom PIL import Imagefrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForImageTextToText, AutoProcessor REPO = "wjbmattingly/comma-lightonocr-3-4b"model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(REPO) # The prompt is part of the model: it was trained to answer this exact instruction.# Use it verbatim -- LightOnOCR's own empty transcription prompt is a different task.prompt = open(hf_hub_download(REPO, "prompt.txt"), encoding="utf-8").read() image = Image.open("page.jpg").convert("RGB")messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3072, do_sample=False, repetition_penalty=1.1, use_cache=True)print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))prompt.txt is required. This model was trained on a 11,629-character instruction,
shipped in this repo; LightOnOCR-3 itself was trained on an empty prompt and emits
block-labelled markdown with bounding boxes, which is not this task.enable_thinking=False. Passed explicitly so the generation prompt closes the
thinking block regardless of which template variant is loaded.processor_config.json is capped at 2,048 visual
tokens, the budget used in training and scoring. LightOnOCR-3 ships uncapped
(~16k tokens); loading a stock processor shows the page at a resolution this model
never trained on.repetition_penalty=1.1 is this project's measured serving setting: it removes most
looping on sparse pages at a small cost in line recall. The table above is greedy
(penalty 1.0); set 1.0 to reproduce it.generation_config.json sets use_cache: true. LightOnOCR-3-0.8B ships false (a
training artefact), which makes generation recompute the whole sequence at every
step; every number above was produced with the cache on.Full fine-tune of all 4,539,265,536 parameters (vision tower included), 3 epochs, 3,717 steps, learning rate 5e-05, cosine schedule, effective batch 8, bf16, 2,048 visual tokens per page, on H200. 2 of 9,906 pages exceeded the 8,192-token length budget and were dropped (the stock-Qwen control dropped 1).
Data: comma-project/deep-jsonl.
A research artifact for cultural-heritage and digital-humanities work, not a production OCR service. Trained on one collection, it should be expected to degrade on other hands, languages, genres and centuries. It reproduces what is written, including historical spelling and language; transcribing a record is not an endorsement of its contents. Verify any name, date or figure before relying on it.
import torchfrom PIL import Imagefrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForImageTextToText, AutoProcessor REPO = "wjbmattingly/comma-lightonocr-3-4b"model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(REPO) # The prompt is part of the model: it was trained to answer this exact instruction.# Use it verbatim -- LightOnOCR's own empty transcription prompt is a different task.prompt = open(hf_hub_download(REPO, "prompt.txt"), encoding="utf-8").read() image = Image.open("page.jpg").convert("RGB")messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3072, do_sample=False, repetition_penalty=1.1, use_cache=True)print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))