GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
orgessnk
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
orgessnk
Model Tree
allenai/olmOCR-2-7B-1025
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom transformers import AutoModelForVision2Seq, AutoProcessor, BitsAndBytesConfigfrom PIL import Image # Configure 4-bit loadingquantization_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True) # Load modelmodel = AutoModelForVision2Seq.from_pretrained( "orgessnk/olmOCR-2-7B-1025-BNB-4bit", quantization_config=quantization_config, device_map="auto", trust_remote_code=True) processor = AutoProcessor.from_pretrained( "orgessnk/olmOCR-2-7B-1025-BNB-4bit", trust_remote_code=True) # Perform OCRimage = Image.open("document.jpg")messages = [ { "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": "Extract all text from this image."} ] }] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)from qwen_vl_utils import process_vision_infoimage_inputs, video_inputs = process_vision_info(messages) inputs = processor( text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt").to("cuda") outputs = model.generate(**inputs, max_new_tokens=512)result = processor.batch_decode(outputs, skip_special_tokens=True)[0]print(result)bash
pip install torch transformers accelerate bitsandbytes qwen-vl-utilsThis model was quantized using bitsandbytes with the following configuration:
NF4 quantization is specifically designed for neural network weights and provides better quality than standard 4-bit quantization.
Same as base model: Apache 2.0
If you use this model, please cite the original olmOCR paper:
bibtex
@article{olmocr2024, title={olmOCR: Open Large Multimodal Model for OCR}, author={AllenAI Team}, year={2024}}import torchfrom transformers import AutoModelForVision2Seq, AutoProcessor, BitsAndBytesConfigfrom PIL import Image # Configure 4-bit loadingquantization_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True) # Load modelmodel = AutoModelForVision2Seq.from_pretrained( "orgessnk/olmOCR-2-7B-1025-BNB-4bit", quantization_config=quantization_config, device_map="auto", trust_remote_code=True) processor = AutoProcessor.from_pretrained( "orgessnk/olmOCR-2-7B-1025-BNB-4bit", trust_remote_code=True) # Perform OCRimage = Image.open("document.jpg")messages = [ { "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": "Extract all text from this image."} ] }] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)from qwen_vl_utils import process_vision_infoimage_inputs, video_inputs = process_vision_info(messages) inputs = processor( text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt").to("cuda") outputs = model.generate(**inputs, max_new_tokens=512)result = processor.batch_decode(outputs, skip_special_tokens=True)[0]print(result)pip install torch transformers accelerate bitsandbytes qwen-vl-utils@article{olmocr2024, title={olmOCR: Open Large Multimodal Model for OCR}, author={AllenAI Team}, year={2024}}