lightonai

LightOnOCR-0.9B-16k-1025

README

License: apache-2.0

Model Overview

LightOnOCR combines a Vision Transformer encoder(Pixtral-based) with a lightweight text decoder(Qwen3-based) distilled from high-quality open VLMs. It is optimized for document parsing tasks, producing accurate, layout-aware text extraction from high-resolution pages.

Benchmarks

Table with columns: Model, ArXiv, Old Scans, Math, Tables, Multi-Column, Tiny Text, Base, Overall
Model	ArXiv	Old Scans	Math	Tables	Multi-Column	Tiny Text	Base	Overall
LightOnOCR-1B-1025 (151k vocab)	81.4	71.6	76.4	35.2	80.0	88.7	99.5	76.1
LightOnOCR-1B-32k (32k vocab)	80.6	66.2	73.5	33.5	71.2	87.6	99.5	73.1
LightOnOCR-1B-16k (16k vocab)	82.3	72.9	75.3	33.5	78.6	85.1	99.8	75.4

All benchmarks evaluated using vLLM.

Installation

[2025/11/24] 🚀 LightOnOCR is now officially supported in vLLM v0.11.1 🚀

bash

uv venv --python 3.12 --seedsource .venv/bin/activate # install any version higher than 0.11.1uv pip install vllm==0.11.2# extra deps need only to run the example belowuv pip install pypdfium2 pillow requests

Start Server

bash

vllm serve lightonai/LightOnOCR-0.9B-16k-1025 \    --limit-mm-per-prompt '{"image": 1}'

PDF Inference

python

import base64import requestsimport pypdfium2 as pdfiumimport io ENDPOINT = "http://localhost:8000/v1/chat/completions"MODEL = "lightonai/LightOnOCR-0.9B-16k-1025" # Download PDF from arXivpdf_url = "https://arxiv.org/pdf/2412.13663"pdf_data = requests.get(pdf_url).content # Open PDF and convert first page to imagepdf = pdfium.PdfDocument(pdf_data)page = pdf[0]# Render at 200 DPI (scale factor = 200/72 ≈ 2.77)pil_image = page.render(scale=2.77).to_pil() # Convert to base64buffer = io.BytesIO()pil_image.save(buffer, format="PNG")image_base64 = base64.b64encode(buffer.getvalue()).decode('utf-8') # Make requestpayload = {    "model": MODEL,    "messages": [{        "role": "user",        "content": [{            "type": "image_url",            "image_url": {"url": f"data:image/png;base64,{image_base64}"}        }]    }],    "max_tokens": 4096,    "temperature": 0.2,    "top_p": 0.9,} response = requests.post(ENDPOINT, json=payload)text = response.json()['choices'][0]['message']['content']print(text)

Rendering and Preprocessing Tips

Render PDFs to PNG or JPEG at a target longest dimension of 1280–1300 px
Maintain aspect ratio to preserve text geometry
LightOnOCR is robust to moderate skew; heavy rotation correction is optional
Use one image per page; batching supported by vLLM

Variants

Table with columns: Variant, Description
Variant	Description
LightOnOCR-1B-1025	Full multilingual model (default)
LightOnOCR-1B-32k	Fastest pruned-vocabulary version (32k tokens) optimized for European languages
LightOnOCR-1B-16k	Most compact variant with smallest vocabulary

Fine-tuning

Transformers integration is coming soon for training.

LightOnOCR is fully differentiable and supports:

LoRA fine-tuning
Domain adaptation (receipts, scientific articles, forms, etc.)
Multilingual fine-tuning with task-specific corpora

Example fine-tuning configurations will be released alongside the dataset.

Data

Trained on a diverse large-scale PDF corpus covering:

Scientific papers, books, receipts, invoices, tables, forms, and handwritten text
Multiple languages (Latin alphabet dominant)
Real and synthetic document scans

The dataset will be released under an open license.

License

Apache License 2.0

Acknowlegments

The project received funding from the BPI Scribe project.

Citation

text

@misc{lightonocr2025,  title        = {LightOnOCR-1B: End-to-End and Efficient Domain-Specific Vision-Language Models for OCR},  author       = {Said Taghadouini and Baptiste Aubertin and Adrien Cavaillès},  year         = {2025},  howpublished = {\url{https://huggingface.co/blog/lightonai/lightonocr}}}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

lightonai

Model Tree

Base

lightonai/LightOnOCR-1B-1025

Fine-tuned

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Overview

Benchmarks

Table with columns: Model, ArXiv, Old Scans, Math, Tables, Multi-Column, Tiny Text, Base, Overall
Model	ArXiv	Old Scans	Math	Tables	Multi-Column	Tiny Text	Base	Overall
LightOnOCR-1B-1025 (151k vocab)	81.4	71.6	76.4	35.2	80.0	88.7	99.5	76.1
LightOnOCR-1B-32k (32k vocab)	80.6	66.2	73.5	33.5	71.2	87.6	99.5	73.1
LightOnOCR-1B-16k (16k vocab)	82.3	72.9	75.3	33.5	78.6	85.1	99.8	75.4

All benchmarks evaluated using vLLM.

Installation

[2025/11/24] 🚀 LightOnOCR is now officially supported in vLLM v0.11.1 🚀

bash

uv venv --python 3.12 --seedsource .venv/bin/activate # install any version higher than 0.11.1uv pip install vllm==0.11.2# extra deps need only to run the example belowuv pip install pypdfium2 pillow requests

uv venv --python 3.12 --seedsource .venv/bin/activate # install any version higher than 0.11.1uv pip install vllm==0.11.2# extra deps need only to run the example belowuv pip install pypdfium2 pillow requests

Start Server

bash

vllm serve lightonai/LightOnOCR-0.9B-16k-1025 \    --limit-mm-per-prompt '{"image": 1}'

vllm serve lightonai/LightOnOCR-0.9B-16k-1025 \    --limit-mm-per-prompt '{"image": 1}'

PDF Inference

python

import base64import requestsimport pypdfium2 as pdfiumimport io ENDPOINT = "http://localhost:8000/v1/chat/completions"MODEL = "lightonai/LightOnOCR-0.9B-16k-1025" # Download PDF from arXivpdf_url = "https://arxiv.org/pdf/2412.13663"pdf_data = requests.get(pdf_url).content # Open PDF and convert first page to imagepdf = pdfium.PdfDocument(pdf_data)page = pdf[0]# Render at 200 DPI (scale factor = 200/72 ≈ 2.77)pil_image = page.render(scale=2.77).to_pil() # Convert to base64buffer = io.BytesIO()pil_image.save(buffer, format="PNG")image_base64 = base64.b64encode(buffer.getvalue()).decode('utf-8') # Make requestpayload = {    "model": MODEL,    "messages": [{        "role": "user",        "content": [{            "type": "image_url",            "image_url": {"url": f"data:image/png;base64,{image_base64}"}        }]    }],    "max_tokens": 4096,    "temperature": 0.2,    "top_p": 0.9,} response = requests.post(ENDPOINT, json=payload)text = response.json()['choices'][0]['message']['content']print(text)

import base64import requestsimport pypdfium2 as pdfiumimport io ENDPOINT = "http://localhost:8000/v1/chat/completions"MODEL = "lightonai/LightOnOCR-0.9B-16k-1025" # Download PDF from arXivpdf_url = "https://arxiv.org/pdf/2412.13663"pdf_data = requests.get(pdf_url).content # Open PDF and convert first page to imagepdf = pdfium.PdfDocument(pdf_data)page = pdf[0]# Render at 200 DPI (scale factor = 200/72 ≈ 2.77)pil_image = page.render(scale=2.77).to_pil() # Convert to base64buffer = io.BytesIO()pil_image.save(buffer, format="PNG")image_base64 = base64.b64encode(buffer.getvalue()).decode('utf-8') # Make requestpayload = {    "model": MODEL,    "messages": [{        "role": "user",        "content": [{            "type": "image_url",            "image_url": {"url": f"data:image/png;base64,{image_base64}"}        }]    }],    "max_tokens": 4096,    "temperature": 0.2,    "top_p": 0.9,} response = requests.post(ENDPOINT, json=payload)text = response.json()['choices'][0]['message']['content']print(text)

Rendering and Preprocessing Tips

Render PDFs to PNG or JPEG at a target longest dimension of 1280–1300 px
Maintain aspect ratio to preserve text geometry
LightOnOCR is robust to moderate skew; heavy rotation correction is optional
Use one image per page; batching supported by vLLM

Variants

Table with columns: Variant, Description
Variant	Description
LightOnOCR-1B-1025	Full multilingual model (default)
LightOnOCR-1B-32k	Fastest pruned-vocabulary version (32k tokens) optimized for European languages
LightOnOCR-1B-16k	Most compact variant with smallest vocabulary

Fine-tuning

Transformers integration is coming soon for training.

LightOnOCR is fully differentiable and supports:

LoRA fine-tuning
Domain adaptation (receipts, scientific articles, forms, etc.)
Multilingual fine-tuning with task-specific corpora

Example fine-tuning configurations will be released alongside the dataset.

Data

Trained on a diverse large-scale PDF corpus covering:

Scientific papers, books, receipts, invoices, tables, forms, and handwritten text
Multiple languages (Latin alphabet dominant)
Real and synthetic document scans

The dataset will be released under an open license.

License

Apache License 2.0

Acknowlegments

The project received funding from the BPI Scribe project.

Citation

text

@misc{lightonocr2025,  title        = {LightOnOCR-1B: End-to-End and Efficient Domain-Specific Vision-Language Models for OCR},  author       = {Said Taghadouini and Baptiste Aubertin and Adrien Cavaillès},  year         = {2025},  howpublished = {\url{https://huggingface.co/blog/lightonai/lightonocr}}}

@misc{lightonocr2025,  title        = {LightOnOCR-1B: End-to-End and Efficient Domain-Specific Vision-Language Models for OCR},  author       = {Said Taghadouini and Baptiste Aubertin and Adrien Cavaillès},  year         = {2025},  howpublished = {\url{https://huggingface.co/blog/lightonai/lightonocr}}}