GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
larshiakzemil
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
larshiakzemil
Model Tree
allenai/olmOCR-2-7B-1025
Input Modalities
Output Modalities
Supported Functionality
| Item | Value |
|---|---|
| Base model | allenai/olmOCR-2-7B-1025 (~7B) |
| Method | QLoRA (4-bit base + LoRA adapters) |
| LoRA rank / alpha | r=16, alpha=32, dropout 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj |
| Trainable params | ~10.1M (0.12% of 8.3B) |
| Hardware | 1× RTX 3090 24GB |
| Train / eval | 159,999 / 2,000 page pairs (80/20) |
| Batch | 4 (effective 4), image longest side 768 |
| Learning rate | 2e-4 cosine, warmup 5% |
| Planned epochs | 100 (stopped early ~epoch 2.0) |
| Best checkpoint | checkpoint-80000 |
| Best eval_loss | 0.0487 |
| Train runtime (to best) | ~63h |
| Step | eval_loss |
|---|---|
| 10,000 | 0.0963 |
| 20,000 | 0.0841 |
| 30,000 | 0.0736 |
| 40,000 | 0.0665 |
| 50,000 | 0.0586 |
| 60,000 | 0.0559 |
| 70,000 | 0.0497 |
| 80,000 | 0.0487 |
Eval loss kept falling through step 80k; this upload is the best (and latest) saved adapter from that run.
adapter_model.safetensors — LoRA weights onlyadapter_config.json — PEFT config (base_model_name_or_path → allenai/olmOCR-2-7B-1025)Optimizer / scheduler / training state were not uploaded.
python
import torchfrom PIL import Imagefrom peft import PeftModelfrom transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration base_id = "allenai/olmOCR-2-7B-1025"lora_id = "larshiakzemil/olmocr-farsi-lora"device = "cuda" if torch.cuda.is_available() else "cpu"dtype = torch.bfloat16 if device == "cuda" else torch.float32 processor = AutoProcessor.from_pretrained(base_id, trust_remote_code=True)model = Qwen2_5_VLForConditionalGeneration.from_pretrained( base_id, torch_dtype=dtype, device_map=device, trust_remote_code=True)model = PeftModel.from_pretrained(model, lora_id)model.eval() image = Image.open("page.jpg").convert("RGB")prompt = ( "Attached is one page of a document that you must process. " "Just return the plain text representation of this document as if you were reading it naturally. " "Convert equations to LateX and tables to HTML.\n" "Return your output as markdown, with a front matter section on top specifying values for the " "primary_language, is_rotation_valid, rotation_correction, is_table, and is_diagram parameters.")messages = [{ "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": prompt}, ],}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[text], images=[image], return_tensors="pt")inputs = {k: v.to(device) if hasattr(v, "to") else v for k, v in inputs.items()} with torch.inference_mode(): out = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(processor.decode(out[0], skip_special_tokens=True))Prefer the official olmOCR v4 YAML prompt (build_no_anchoring_v4_yaml_prompt) when using the olmOCR toolkit — that matches training.
larshiakzemil/glm-ocr-farsi.import torchfrom PIL import Imagefrom peft import PeftModelfrom transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration base_id = "allenai/olmOCR-2-7B-1025"lora_id = "larshiakzemil/olmocr-farsi-lora"device = "cuda" if torch.cuda.is_available() else "cpu"dtype = torch.bfloat16 if device == "cuda" else torch.float32 processor = AutoProcessor.from_pretrained(base_id, trust_remote_code=True)model = Qwen2_5_VLForConditionalGeneration.from_pretrained( base_id, torch_dtype=dtype, device_map=device, trust_remote_code=True)model = PeftModel.from_pretrained(model, lora_id)model.eval() image = Image.open("page.jpg").convert("RGB")prompt = ( "Attached is one page of a document that you must process. " "Just return the plain text representation of this document as if you were reading it naturally. " "Convert equations to LateX and tables to HTML.\n" "Return your output as markdown, with a front matter section on top specifying values for the " "primary_language, is_rotation_valid, rotation_correction, is_table, and is_diagram parameters.")messages = [{ "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": prompt}, ],}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[text], images=[image], return_tensors="pt")inputs = {k: v.to(device) if hasattr(v, "to") else v for k, v in inputs.items()} with torch.inference_mode(): out = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(processor.decode(out[0], skip_special_tokens=True))