GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
NeuralMetrics
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
NeuralMetrics
Model Tree
Qwen/Qwen2-VL-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This model expects as input a single document image, rendered such that the longest dimension is 1024 pixels.
The prompt must then contain the additional metadata from the document, and the easiest way to generate this is to use the methods provided by the olmOCR toolkit.
If you want to prompt this model manually instead of using the olmOCR toolkit, please see the code below.
In normal usage, the olmOCR toolkit builds the prompt by rendering the PDF page, and extracting relevant text blocks and image metadata. To duplicate that you will need to
bash
pip install olmocrand then run the following sample code.
python
import torchimport base64import urllib.request from io import BytesIOfrom PIL import Imagefrom transformers import AutoProcessor, Qwen2VLForConditionalGeneration from olmocr.data.renderpdf import render_pdf_to_base64pngfrom olmocr.prompts import build_finetuning_promptfrom olmocr.prompts.anchor import get_anchor_text # Initialize the modelmodel = Qwen2VLForConditionalGeneration.from_pretrained("allenai/olmOCR-7B-0225-preview", torch_dtype=torch.bfloat16).eval()processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-7B-Instruct")device = torch.device("cuda" if torch.cuda.is_available() else "cpu")model.to(device) # Grab a sample PDFurllib.request.urlretrieve("https://molmo.allenai.org/paper.pdf", "./paper.pdf") # Render page 1 to an imageimage_base64 = render_pdf_to_base64png("./paper.pdf", 1, target_longest_image_dim=1024) # Build the prompt, using document metadataanchor_text = get_anchor_text("./paper.pdf", 1, pdf_engine="pdfreport", target_length=4000)prompt = build_finetuning_prompt(anchor_text) # Build the full promptmessages = [ { "role": "user", "content": [ {"type": "text", "text": prompt}, {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_base64}"}}, ], } ] # Apply the chat template and processortext = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)main_image = Image.open(BytesIO(base64.b64decode(image_base64))) inputs = processor( text=[text], images=[main_image], padding=True, return_tensors="pt",)inputs = {key: value.to(device) for (key, value) in inputs.items()} # Generate the outputoutput = model.generate( **inputs, temperature=0.8, max_new_tokens=50, num_return_sequences=1, do_sample=True, ) # Decode the outputprompt_length = inputs["input_ids"].shape[1]new_tokens = output[:, prompt_length:]text_output = processor.tokenizer.batch_decode( new_tokens, skip_special_tokens=True) print(text_output)# ['{"primary_language":"en","is_rotation_valid":true,"rotation_correction":0,"is_table":false,"is_diagram":false,"natural_text":"Molmo and PixMo:\\nOpen Weights and Open Data\\nfor State-of-the']olmOCR is licensed under the Apache 2.0 license. olmOCR is intended for research and educational use. For more information, please see our Responsible Use Guidelines.
import torchimport base64import urllib.request from io import BytesIOfrom PIL import Imagefrom transformers import AutoProcessor, Qwen2VLForConditionalGeneration from olmocr.data.renderpdf import render_pdf_to_base64pngfrom olmocr.prompts import build_finetuning_promptfrom olmocr.prompts.anchor import get_anchor_text # Initialize the modelmodel = Qwen2VLForConditionalGeneration.from_pretrained("allenai/olmOCR-7B-0225-preview", torch_dtype=torch.bfloat16).eval()processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-7B-Instruct")device = torch.device("cuda" if torch.cuda.is_available() else "cpu")model.to(device) # Grab a sample PDFurllib.request.urlretrieve("https://molmo.allenai.org/paper.pdf", "./paper.pdf") # Render page 1 to an imageimage_base64 = render_pdf_to_base64png("./paper.pdf", 1, target_longest_image_dim=1024) # Build the prompt, using document metadataanchor_text = get_anchor_text("./paper.pdf", 1, pdf_engine="pdfreport", target_length=4000)prompt = build_finetuning_prompt(anchor_text) # Build the full promptmessages = [ { "role": "user", "content": [ {"type": "text", "text": prompt}, {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_base64}"}}, ], } ] # Apply the chat template and processortext = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)main_image = Image.open(BytesIO(base64.b64decode(image_base64))) inputs = processor( text=[text], images=[main_image], padding=True, return_tensors="pt",)inputs = {key: value.to(device) for (key, value) in inputs.items()} # Generate the outputoutput = model.generate( **inputs, temperature=0.8, max_new_tokens=50, num_return_sequences=1, do_sample=True, ) # Decode the outputprompt_length = inputs["input_ids"].shape[1]new_tokens = output[:, prompt_length:]text_output = processor.tokenizer.batch_decode( new_tokens, skip_special_tokens=True) print(text_output)# ['{"primary_language":"en","is_rotation_valid":true,"rotation_correction":0,"is_table":false,"is_diagram":false,"natural_text":"Molmo and PixMo:\\nOpen Weights and Open Data\\nfor State-of-the']