GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ZenithComp
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ZenithComp
Model Tree
typhoon-ai/typhoon-ocr-7b
Input Modalities
Output Modalities
Supported Functionality
typhoon-ai/typhoon-ocr-7b (Qwen2.5-VL-7B, Apache-2.0)max_pixels=1048576), greedy decoding
(do_sample=False), max_new_tokens=512. Other regimes are outside the evaluation scope.This is the evaluation attached to the branch promoted to main:
phase2-downsample-money-keep25-300steps, evaluated on the 559-sample CPE-OPH full test with
1024px images, greedy decoding, and max_new_tokens=512.
| Metric | Promotion baseline | Promoted main | Delta | Direction |
|---|---|---|---|---|
valid_json_rate | 1.0000 | 1.0000 | +0.0000 | higher is better |
raw_cer | 0.1560 | 0.1336 | -0.0224 | lower is better |
normalized_cer | 0.1560 | 0.1336 | -0.0224 | lower is better |
thai_char_accuracy | 0.8440 | 0.8664 | +0.0224 | higher is better |
exact_match_rate | 0.4204 | 0.4526 | +0.0322 | higher is better |
Baseline: phase2-baseline-greedy-res1024. Source:
published metrics artifact.
The CPE-OPH full-test result is not a clean generalization score because gold texts overlap between its train and test splits.
Install dependencies first (qwen-vl-utils is a separate package — hyphens in the pip name,
underscores in the import):
bash
pip install "transformers>=4.49" accelerate peft torch pillow qwen-vl-utilspython
import json, torchfrom PIL import Imagefrom transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom peft import PeftModelfrom qwen_vl_utils import process_vision_info BASE = "typhoon-ai/typhoon-ocr-7b"ADAPTER = "sivakorn-su/typhoon-ocr-7b-thai-handwriting-lora-v1" # the `main` adapter processor = AutoProcessor.from_pretrained(BASE, max_pixels=1048576) # 1024px regimemodel = Qwen2_5_VLForConditionalGeneration.from_pretrained(BASE, torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER)model.eval() PROMPT = ( "Below is an image of a document page along with its dimensions. " "Simply return the markdown representation of this document, presenting tables in markdown format as they naturally appear.\n" "If the document contains images, use a placeholder like dummy.png for each image.\n" "Your final output must be in JSON format with a single key `natural_text` containing the response.\n" "RAW_TEXT_START\n\nRAW_TEXT_END") image = Image.open("handwriting.png").convert("RGB")messages = [{"role": "user", "content": [ {"type": "text", "text": PROMPT}, {"type": "image", "image": image},]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)image_inputs, video_inputs = process_vision_info(messages)inputs = processor(text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=512, do_sample=False) # greedydecoded = processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0].strip()print(json.loads(decoded)["natural_text"])Prompt note (important): the prompt above is the exact one this adapter was fine-tuned with — Typhoon's
defaulttask template with an emptyRAW_TEXTanchor and no injected page dimensions. Base Typhoon normally runs via thetyphoon-ocrpackage (ocr_document), which builds a richer prompt (OCR anchor text fromget_anchor_text+ real dimensions). Because this adapter was trained on the simplified, empty-anchor prompt, use it as-is — do not substitute the package's full prompt, or inference will mismatch training. (Typhoon: the model only works with its specific prompt templates.)
The Performance table reports only the CPE-OPH evaluation used to promote this adapter to main.
Phase 3 evaluations belong to their experimental branches and are intentionally excluded here.
The result does not establish performance on full document forms, layouts, unseen writers, or
capture conditions outside the evaluation set.
Promoted to main from branch phase2-downsample-money-keep25-300steps after full-test evaluation and
promotion-gate checks. Base: typhoon-ai/typhoon-ocr-7b; trained and evaluated
on Thinnaphat/TH-HANDWRITTEN-CPE-OPH2025.
Apache-2.0, inheriting the base model typhoon-ai/typhoon-ocr-7b → Qwen/Qwen2.5-VL-7B-Instruct.
Training data: Thinnaphat/TH-HANDWRITTEN-CPE-OPH2025 (a subset of iapp/thai_handwriting_dataset,
Apache-2.0). This repository is a LoRA adapter only.
pip install "transformers>=4.49" accelerate peft torch pillow qwen-vl-utilsimport json, torchfrom PIL import Imagefrom transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom peft import PeftModelfrom qwen_vl_utils import process_vision_info BASE = "typhoon-ai/typhoon-ocr-7b"ADAPTER = "sivakorn-su/typhoon-ocr-7b-thai-handwriting-lora-v1" # the `main` adapter processor = AutoProcessor.from_pretrained(BASE, max_pixels=1048576) # 1024px regimemodel = Qwen2_5_VLForConditionalGeneration.from_pretrained(BASE, torch_dtype="auto", device_map="auto")model = PeftModel.from_pretrained(model, ADAPTER)model.eval() PROMPT = ( "Below is an image of a document page along with its dimensions. " "Simply return the markdown representation of this document, presenting tables in markdown format as they naturally appear.\n" "If the document contains images, use a placeholder like dummy.png for each image.\n" "Your final output must be in JSON format with a single key `natural_text` containing the response.\n" "RAW_TEXT_START\n\nRAW_TEXT_END") image = Image.open("handwriting.png").convert("RGB")messages = [{"role": "user", "content": [ {"type": "text", "text": PROMPT}, {"type": "image", "image": image},]}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)image_inputs, video_inputs = process_vision_info(messages)inputs = processor(text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=512, do_sample=False) # greedydecoded = processor.batch_decode(out[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0].strip()print(json.loads(decoded)["natural_text"])