GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
nanash66
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
nanash66
Model Tree
typhoon-ai/typhoon-ocr1.5-2b
Input Modalities
Output Modalities
Supported Functionality
| Property | Value |
|---|---|
| Base Model | typhoon-ai/typhoon-ocr1.5-2b |
| Architecture | Qwen3VLForConditionalGeneration |
| Quantization | NVFP4 (W4A4), group_size=16 |
| Quantizer | NVIDIA ModelOpt 0.45.0 |
| Target GPU | NVIDIA Blackwell (GB10 / sm_121) |
| Quantized Size | 2.23 GB |
| Context Length | 262,144 tokens |
| Hidden Size | 2048 |
| Layers | 28 |
| Attention Heads | 16 (KV: 8) |
| Vision | Qwen3VL Vision Encoder (1024 dim, 24 layers) |
lm_head, model.language_model.embed_tokens, model.visual*| File | Size |
|---|---|
model.safetensors | 2.23 GB |
config.json | Model config with quantization_config |
hf_quant_config.json | HF quantization metadata |
generation_config.json | Generation defaults |
chat_template.jinja | Chat template |
tokenizer.json |
This is a task-specific model intended to be used only with the provided prompt. It does not include any guardrails or VQA capability. Due to the nature of large language models (LLMs), a certain level of hallucination may occur. We recommend that developers carefully assess these risks in the context of their specific use case.
Important: This model is a single-prompt model — it works with one specific extraction prompt of Typhoon OCR v1.5 only. Other prompts will not produce correct results.
The official extraction prompt for Typhoon OCR v1.5:
python
prompt = """Extract all text from the image. Instructions:- Only return the clean Markdown.- Do not include any explanation or extra text.- You must include all information on the page. Formatting Rules:- Tables: Render tables using <table>...</table> in clean HTML format.- Equations: Render equations using LaTeX syntax with inline ($...$) and block ($$...$$).- Images/Charts/Diagrams: Wrap any clearly defined visual areas (e.g. charts, diagrams, pictures) in: <figure>Describe the image's main elements (people, objects, text), note any contextual clues (place, event, culture), mention visible text and its meaning, provide deeper analysis when relevant (especially for financial charts, graphs, or documents), comment on style or architecture if relevant, then give a concise overall summary. Describe in Thai.</figure> - Page Numbers: Wrap page numbers in <page_number>...</page_number> (e.g., <page_number>14</page_number>).- Checkboxes: Use ☐ for unchecked and ☑ for checked boxes."""typhoon-ocr libraryWe recommend running inference via vLLM with the typhoon-ocr library rather than raw transformers:
bash
pip install vllm typhoon-ocrServe the local NVFP4 model:
bash
vllm serve nanash66/typhoon-ocr1.5-2b-NVFP4 \ --quantization modelopt \ --max-model-len 49152 \ --served-model-name typhoon-ocr-nvfp4python
from typhoon_ocr import ocr_document markdown = ocr_document( "image.png", model="typhoon-ocr-nvfp4", figure_language="Thai", task_type="v1.5", base_url="http://localhost:8000/v1", api_key="no-key",)print(markdown)Note: This model requires a Blackwell GPU (sm_121) with NVIDIA ModelOpt runtime for NVFP4 inference.
python
from transformers import AutoModelForImageTextToText, AutoProcessorfrom PIL import Image model = AutoModelForImageTextToText.from_pretrained( "nanash66/typhoon-ocr1.5-2b-NVFP4", dtype="auto", device_map="auto",)processor = AutoProcessor.from_pretrained("nanash66/typhoon-ocr1.5-2b-NVFP4") # Load imageimg = Image.open("image.png") # IMPORTANT: The model was trained at a fixed image dimension of 1800px.# Resize the image before inference.def resize_if_needed(img, max_size=1800): if max(img.size) > max_size: scale = max_size / max(img.size) img = img.resize((int(img.width * scale), int(img.height * scale))) return img img = resize_if_needed(img, 1800) messages = [ { "role": "user", "content": [ {"type": "image", "image": img}, {"type": "text", "text": prompt}, ], }] # Preparation for inferenceinputs = processor.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",).to(model.device) # Inferencegenerated_ids = model.generate(**inputs, max_new_tokens=10000)generated_ids_trimmed = [ out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]output_text = processor.batch_decode( generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False,)print(output_text[0])This is a derivative work of typhoon-ai/typhoon-ocr1.5-2b, released under the Apache-2.0 license.
If you find this model useful, please cite the original works:
bibtex
@misc{nonesung2026typhoonocropenvisionlanguage, title={Typhoon OCR: Open Vision-Language Model For Thai Document Extraction}, author={Surapon Nonesung and Natapong Nitarach and Teetouch Jaknamon and Pittawat Taveekitworachai and Kunat Pipatanakul}, year={2026}, eprint={2601.14722}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2601.14722},} @misc{typhoon2, title={Typhoon 2}, year={2024}, eprint={2412.13702}, archivePrefix={arXiv}, url={https://arxiv.org/abs/2412.13702},} @misc{thaiocrbench, title={ThaiOCRBench}, year={2025}, eprint={2511.04479}, archivePrefix={arXiv}, url={https://arxiv.org/abs/2511.04479},}Apache-2.0 (same as base model)
| Tokenizer model |
tokenizer_config.json | Tokenizer config |
preprocessor_config.json | Vision preprocessor config |
video_preprocessor_config.json | Video preprocessor config |