GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
wjbmattingly
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
wjbmattingly
Model Tree
lightonai/LightOnOCR-3-0.8B
Input Modalities
Output Modalities
Supported Functionality
762 held-out pages from 39 documents, none of which appear among the 1,407 training documents. Documents are held out whole: ~27 pages of one document share a hand and a printed form, so a page-level split would measure memorisation. NFD normalisation is a no-op on English ASCII, so plain CER is quoted.
| model | CER | macro | WER | line recall | CER excl. degenerate | truncated |
|---|---|---|---|---|---|---|
| LightOnOCR-3-0.8B, zero-shot (same job) | 0.4185 | 2.5225 | 0.5283 | 0.1697 | 0.3953 | 2.5% |
| this model | 0.2550 | 0.4133 | 0.3297 | 0.1867 | 0.2230 | 1.0% |
| LightOnOCR-3-4B, zero-shot | 0.2057 | 0.3259 | 0.2794 | 0.1956 | 0.2057 | 0.1% |
| LightOnOCR-3-4B, full FT | 0.2312 | 0.3531 | 0.2964 | 0.1927 | 0.1857 | 0.7% |
| nara-qwen-3.5-2b (published) | 0.2688 | 0.4607 | 0.3205 | 0.1911 | 0.2688 | 0.9% |
| nara-qwen-3.5-0.8b (published) | 0.2868 | 0.3822 | 0.3379 | 0.1901 | 0.2328 | 1.3% |
Greedy decoding, 3,072 max new tokens, 2,048 visual tokens per page, eval batch 4. Treat differences under 0.02 CER as ties — four seeds of one configuration in this project span 0.021.
It beats the 0.8B NARA model this project published before it, and matches the 2B,
from a simpler recipe. 0.2550 CER against 0.2868 for nara-qwen-3.5-0.8b — a gap of
0.032, outside the noise floor, on the same 762 pages scored on the same GPU type.
Against nara-qwen-3.5-2b (0.2688) the gap is 0.014, inside the noise floor:
a tie. Those earlier models needed a three-stage curriculum (medieval Latin, then
10,000 NARA rows, then 40,000); this one is a single three-epoch pass from LightOn's
checkpoint.
The zero-shot macro CER of 2.5225 is almost entirely one page. A card whose
reference is 213.214 (7 characters) drew 10,499 characters of output — a
hallucinated The United States National Archives… Page 1 of 1 footer, looped to the
token ceiling — for a per-page CER of 1,499, which alone contributes 1.97 of the
2.5225 average. Macro CER weights every page equally, so one runaway page on a
7-character card dominates it.
But if you can afford a 4B, its untrained base is better still. Stock
LightOnOCR-3-4B given this repo's prompt.txt scores 0.2057 with no NARA training at all.
The comparison crosses GPU types, so read the exact gap loosely — but the direction
is not in doubt.
This model and the comparison rows were scored on RTX PRO 6000 Blackwell GPUs, except both LightOnOCR-3-4B rows, which were scored on H200s. That matters: the same untrained LightOnOCR-3-4B scored 0.0275 CER apart on those two GPU types (bf16 arithmetic differs and greedy decoding amplifies it), so gaps under ~0.03 between this model and the H200 rows are ties. The comparisons with the zero-shot row and the RTX-scored rows are like-for-like.
python
import torchfrom PIL import Imagefrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForImageTextToText, AutoProcessor REPO = "wjbmattingly/nara-lightonocr-3-0.8b"model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(REPO) # The prompt is part of the model: it was trained to answer this exact instruction.# Use it verbatim -- LightOnOCR's own empty transcription prompt is a different task.prompt = open(hf_hub_download(REPO, "prompt.txt"), encoding="utf-8").read() image = Image.open("page.jpg").convert("RGB")messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3072, do_sample=False, repetition_penalty=1.1, use_cache=True)print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))prompt.txt is required. This model was trained on a 1,539-character instruction,
shipped in this repo; LightOnOCR-3 itself was trained on an empty prompt and emits
block-labelled markdown with bounding boxes, which is not this task.enable_thinking=False. Passed explicitly so the generation prompt closes the
thinking block regardless of which template variant is loaded.processor_config.json is capped at 2,048 visual
tokens, the budget used in training and scoring. LightOnOCR-3 ships uncapped
(~16k tokens); loading a stock processor shows the page at a resolution this model
never trained on.repetition_penalty=1.1 is this project's measured serving setting: it removes most
looping on sparse pages at a small cost in line recall. The table above is greedy
(penalty 1.0); set 1.0 to reproduce it.generation_config.json sets use_cache: true. LightOnOCR-3-0.8B ships false (a
training artefact), which makes generation recompute the whole sequence at every
step; every number above was produced with the cache on.Full fine-tune of all 852,985,920 parameters (vision tower included), 3 epochs, 14,535 steps, learning rate 5e-05, cosine schedule, effective batch 8, bf16, 2,048 visual tokens per page, on RTX PRO 6000 Blackwell.
Data: wjbmattingly/si-test (first 40,000 rows).
A research artifact for cultural-heritage and digital-humanities work, not a production OCR service. Trained on one collection, it should be expected to degrade on other hands, languages, genres and centuries. It reproduces what is written, including historical spelling and language; transcribing a record is not an endorsement of its contents. Verify any name, date or figure before relying on it.
import torchfrom PIL import Imagefrom huggingface_hub import hf_hub_downloadfrom transformers import AutoModelForImageTextToText, AutoProcessor REPO = "wjbmattingly/nara-lightonocr-3-0.8b"model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(REPO) # The prompt is part of the model: it was trained to answer this exact instruction.# Use it verbatim -- LightOnOCR's own empty transcription prompt is a different task.prompt = open(hf_hub_download(REPO, "prompt.txt"), encoding="utf-8").read() image = Image.open("page.jpg").convert("RGB")messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=3072, do_sample=False, repetition_penalty=1.1, use_cache=True)print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))