GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
NeuralMetrics
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
NeuralMetrics
Model Tree
Qwen/Qwen2.5-VL-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
We made three notable changes:
New Base Model: We swapped in a more recent version of the existing model (Qwen2.5-VL-7B) as the foundation.
No Metadata inputs: Unlike the original, we don’t use metadata extracted from PDFs. This significantly reduces prompt length, which in turn lowers both processing time and VRAM usage — without hurting accuracy in most cases.
Rotation of training data: About 15% of the training data was rotated to enhance robustness to off-angle documents. We otherwise use the same training set.
Host your model with vLLM:
bash
export VLLM_USE_V1=1vllm serve reducto/RolmOCR Call the model via openai compatible server:
python
# HOST YOUR OPENAI COMPATIBLE API WITH THE FOLLOWING COMMAND in VLLM:# export VLLM_USE_V1=1# vllm serve reducto/RolmOCR from openai import OpenAIimport base64 client = OpenAI(api_key="123", base_url="http://localhost:8000/v1") model = "reducto/RolmOCR-7b" def encode_image(image_path): with open(image_path, "rb") as image_file: return base64.b64encode(image_file.read()).decode("utf-8") def ocr_page_with_rolm(img_base64): response = client.chat.completions.create( model=model, messages=[ { "role": "user", "content": [ { "type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_base64}"}, }, { "type": "text", "text": "Return the plain text representation of this document as if you were reading it naturally.\n", }, ], } ], temperature=0.2, max_tokens=4096 ) return response.choices[0].message.content test_img_path = "path/to/image.png"img_base64 = encode_image(test_img_path)print(ocr_page_with_rolm(img_base64))text
@misc{RolmOCR, author = {Reducto AI}, title = {RolmOCR: A Faster, Lighter Open Source OCR Model}, year = {2025},}export VLLM_USE_V1=1vllm serve reducto/RolmOCR # HOST YOUR OPENAI COMPATIBLE API WITH THE FOLLOWING COMMAND in VLLM:# export VLLM_USE_V1=1# vllm serve reducto/RolmOCR from openai import OpenAIimport base64 client = OpenAI(api_key="123", base_url="http://localhost:8000/v1") model = "reducto/RolmOCR-7b" def encode_image(image_path): with open(image_path, "rb") as image_file: return base64.b64encode(image_file.read()).decode("utf-8") def ocr_page_with_rolm(img_base64): response = client.chat.completions.create( model=model, messages=[ { "role": "user", "content": [ { "type": "image_url", "image_url": {"url": f"data:image/png;base64,{img_base64}"}, }, { "type": "text", "text": "Return the plain text representation of this document as if you were reading it naturally.\n", }, ], } ], temperature=0.2, max_tokens=4096 ) return response.choices[0].message.content test_img_path = "path/to/image.png"img_base64 = encode_image(test_img_path)print(ocr_page_with_rolm(img_base64))@misc{RolmOCR, author = {Reducto AI}, title = {RolmOCR: A Faster, Lighter Open Source OCR Model}, year = {2025},}