GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
hanji-dev
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
hanji-dev
Model Tree
Qwen/Qwen3-VL-4B-Instruct
Input Modalities
Output Modalities
Supported Functionality
The model returns JSON only: an array of records
json
[{"bbox_2d": [x1, y1, x2, y2], "text_content": "..."}]bbox_2d is [left, top, right, bottom] in normalized 0–1000 page
coordinates (divide by 1000 and multiply by the page width/height to
recover pixel boxes).text_content = "<image>".[x] / [ ] before their label.[].python
from PIL import Image MAX_PIXELS, PATCH = 2_000_000, 32 def preprocess(img: Image.Image) -> Image.Image: w, h = img.size scale = min(1.0, (MAX_PIXELS / (w * h)) ** 0.5) w, h = int(w * scale) // PATCH * PATCH, int(h * scale) // PATCH * PATCH return img.convert("RGB").resize((w, h), Image.LANCZOS)Send the page image followed by exactly this text as the user turn. Do not paraphrase, extend, or reformat it.
text
Detect every BLOCK in this document and return a JSON array. Schema: [{"bbox_2d":[x1,y1,x2,y2], "text_content":"..."}]Coordinates: normalized 0-1000 page coordinates; [x1,y1,x2,y2] = [left,top,right,bottom]. EMPTY PAGE: If the page has no legible content, return exactly []. Otherwise, transcribe every legible content block; a page with only one legible item is not empty. Your DEFAULT is to GROUP. Most such pages form 5-30 records; sparse pages may form only 1-4. More than 30 remains unusual. A"block" is a semantic SECTION (a panel, a heading + its content, a key-value group,or a whole table), NOT a single line, cell, or field. If you are emitting one recordper line, per cell, or per form field, STOP - that is WRONG. When a region is not aclean table, you must STILL group it into section blocks; never fall back toone-record-per-element. Block categories:- Text records: ONE record per block - a heading TOGETHER WITH the lines beneath it, a paragraph, a list, or a key-value field group. text_content = the block's text, with "\n" between its lines. DO NOT emit one record per line.- Table records: a table or dense grid of cells, rendered as GitHub-Flavored Markdown (| col | col |\n|---|---|\n| cell | cell |). Never emit one record per cell or per row - the markdown carries every cell. A repeated item|amount list (receipt lines, menu items) IS a table. HEADERLESS TABLES: if a table has no visible column headings, DO NOT invent any. Render only the visible rows/cells in their observed order. If Markdown syntax needs a separator row, use empty header cells rather than synthetic names like "Column 1". TABLE CELL TEXT: cell contents must be plain visible text. Do NOT add Markdown emphasis or formatting inside cells (no **bold**, _italics_, backticks, or headings) unless that formatting is the only way to preserve information that is visible on the page. TALL TABLES: A logical table on one page is ONE block regardless of row count. Include every visible row in one GFM table. Its bbox_2d must tightly enclose the full table from the first row through the last row.- Image records: ONE record per photo, figure, chart, scan, or non-text graphic. text_content = "<image>". Handwritten signatures, cursive e-signatures, initials, signature scribbles, and signature marks are ALWAYS images - do NOT transcribe or guess them, even if partly readable. Printed labels such as "Signature:" remain text. Do NOT emit for logos < 40 px wide. CRITICAL - transcribe MEANING, not layout glyphs:- Fill-in / blank lines: emit ONLY the label, NOT the blank. Write "Name:" - never "Name:________________". For signature fields, keep the printed label as text and emit the actual signature mark itself as an image record with text_content="<image>".- NEVER reproduce decorative rules or separators - rows of *, -, _, =, ., or any repeated glyph. Omit them entirely; they are not content.- Checkboxes / Y-N / selection fields: write the field and its options on one line with the marks inline - "<row label> Y [x] N [ ]", "<label>: [x] Yes [ ] No"; "[x]" filled, "[ ]" empty; one record per field, not per option.- text_content must equal the VISIBLE text of the block - never pad, repeat, or continue a character run. No single text block exceeds ~20 lines; split a longer section at its sub-headings. Grouping rules:- A section heading and the content beneath it (down to the next heading) form ONE block.- A field label and its value are ONE block - EVEN in a dense report header. Write "Visit Date: 08/12/2025" as one record; NEVER split the label from its value ("Visit Date:" + "08/12/2025" as two records is WRONG).- On forms, when a key has an associated value, merge the key and value into the same block and bbox so the association is explicit. If several related key-value fields are visually grouped, emit the group as one block with one "Label: value" line per field.- A bordered or visually-grouped PANEL (e.g. a "PRESCRIBER INFORMATION" box with all its fields) is ONE block - join its label:value pairs with "\n".- A row of related cells that is NOT a clean table (a lab-result line, a transaction line) is ONE block - join the cells into one line; NEVER one record per cell.- NEVER emit a bare value, a single cell, or a lone field as its own record.- Keep COLUMNS separate: two side-by-side panels (e.g. Patient | Ordering Provider, Bill To | Ship To) are TWO blocks - never merge across the gutter.- DO NOT merge unrelated neighboring panels or sections.- Group by semantic relationship, NOT by bbox size. Small unrelated regions must stay separate. Example: a page title at the top-left and a page number at the top-right are TWO blocks, even if both boxes are small and on the same horizontal band. Only group items that belong to the same section, panel, list, table, or key-value group.- Prefer CORRECT grouping over tight boxes: a block's bbox may be wide and may lightly touch a neighbor - do NOT over-split a section just to keep boxes small or separate.- Page-edge content: if visible text touches or sits near the page boundary, inspect the full edge carefully and make the bbox include the entire visible glyphs/block, even if the box must start at 0 or end at 1000. Do NOT shrink edge boxes inward.- Bbox coverage is strict: every transcribed character in text_content MUST be inside that record's bbox_2d, with no clipped letters. This matters most for small regions, rotated/non-horizontal text, page-edge text, headers/footers, stamps, and fax strips. Use a tight box around the actual region, but never make it so tight that any visible character you transcribed falls outside the box. Output JSON only.Checkboxes: transcribe every checkbox as [x] if marked or [ ] if unmarked, placed before its label (e.g. "[x] Allergies reviewed"; Y/N pairs as "Y [x] N [ ]"). Include every checkbox, including checkbox grids and Y/N option pairs.temperature 0.0, top_p 1.0, no repetition penalty.max_new_tokens = 8192.guided_json in vLLM) with this schema:json
{ "type": "array", "items": { "type": "object", "properties": { "bbox_2d": { "type": "array", "items": {"type": "integer"}, "minItems": 4, "maxItems": 4 }, "text_content": {"type": "string"} }, "required": ["bbox_2d", "text_content"] }}python
import torchfrom PIL import Imagefrom transformers import AutoModelForImageTextToText, AutoProcessor MODEL = "hanji-dev/hanji-parse-4b"model = AutoModelForImageTextToText.from_pretrained( MODEL, dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(MODEL) image = preprocess(Image.open("page.png")) # see preprocessing abovePROMPT = "..." # the exact prompt above messages = [{ "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": PROMPT}, ],}]inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",).to(model.device) out = model.generate(**inputs, max_new_tokens=8192, do_sample=False)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))N-gram speculative decoding gives a large decode speedup on this output format (the repeated JSON keys draft extremely well):
bash
python -m sglang.launch_server \ --model-path hanji-dev/hanji-parse-4b \ --attention-backend fa3 \ --mm-attention-backend fa3 \ --mem-fraction-static 0.85 \ --chunked-prefill-size 8192 \ --speculative-algorithm NGRAM \ --speculative-num-draft-tokens 16 \ --speculative-ngram-max-bfs-breadth 10 \ --enable-deterministic-inference \ --context-length 16384A ready-to-run server that implements the full preprocessing + prompt contract (and a schema-extraction API around it) is available at https://github.com/youlearn-ai/hanji.
Apache-2.0. Fine-tuned from
Qwen/Qwen3-VL-4B-Instruct
(Apache-2.0, © Alibaba Cloud / the Qwen team).
[{"bbox_2d": [x1, y1, x2, y2], "text_content": "..."}]from PIL import Image MAX_PIXELS, PATCH = 2_000_000, 32 def preprocess(img: Image.Image) -> Image.Image: w, h = img.size scale = min(1.0, (MAX_PIXELS / (w * h)) ** 0.5) w, h = int(w * scale) // PATCH * PATCH, int(h * scale) // PATCH * PATCH return img.convert("RGB").resize((w, h), Image.LANCZOS)Detect every BLOCK in this document and return a JSON array.Schema: [{"bbox_2d":[x1,y1,x2,y2], "text_content":"..."}]Coordinates: normalized 0-1000 page coordinates; [x1,y1,x2,y2] = [left,top,right,bottom].EMPTY PAGE: If the page has no legible content, return exactly []. Otherwise, transcribe every legible content block; a page with only one legible item is not empty.Your DEFAULT is to GROUP. Most such pages form 5-30 records; sparse pages may form only 1-4. More than 30 remains unusual. A"block" is a semantic SECTION (a panel, a heading + its content, a key-value group,or a whole table), NOT a single line, cell, or field. If you are emitting one recordper line, per cell, or per form field, STOP - that is WRONG. When a region is not aclean table, you must STILL group it into section blocks; never fall back toone-record-per-element.Block categories:- Text records: ONE record per block - a heading TOGETHER WITH the lines beneath it, a paragraph, a list, or a key-value field group. text_content = the block's text, with "\n" between its lines. DO NOT emit one record per line.- Table records: a table or dense grid of cells, rendered as GitHub-Flavored Markdown (| col | col |\n|---|---|\n| cell | cell |). Never emit one record per cell or per row - the markdown carries every cell. A repeated item|amount list (receipt lines, menu items) IS a table. HEADERLESS TABLES: if a table has no visible column headings, DO NOT invent any. Render only the visible rows/cells in their observed order. If Markdown syntax needs a separator row, use empty header cells rather than synthetic names like "Column 1". TABLE CELL TEXT: cell contents must be plain visible text. Do NOT add Markdown emphasis or formatting inside cells (no **bold**, _italics_, backticks, or headings) unless that formatting is the only way to preserve information that is visible on the page. TALL TABLES: A logical table on one page is ONE block regardless of row count. Include every visible row in one GFM table. Its bbox_2d must tightly enclose the full table from the first row through the last row.- Image records: ONE record per photo, figure, chart, scan, or non-text graphic. text_content = "<image>". Handwritten signatures, cursive e-signatures, initials, signature scribbles, and signature marks are ALWAYS images - do NOT transcribe or guess them, even if partly readable. Printed labels such as "Signature:" remain text. Do NOT emit for logos < 40 px wide.CRITICAL - transcribe MEANING, not layout glyphs:- Fill-in / blank lines: emit ONLY the label, NOT the blank. Write "Name:" - never "Name:________________". For signature fields, keep the printed label as text and emit the actual signature mark itself as an image record with text_content="<image>".- NEVER reproduce decorative rules or separators - rows of *, -, _, =, ., or any repeated glyph. Omit them entirely; they are not content.- Checkboxes / Y-N / selection fields: write the field and its options on one line with the marks inline - "<row label> Y [x] N [ ]", "<label>: [x] Yes [ ] No"; "[x]" filled, "[ ]" empty; one record per field, not per option.- text_content must equal the VISIBLE text of the block - never pad, repeat, or continue a character run. No single text block exceeds ~20 lines; split a longer section at its sub-headings.Grouping rules:- A section heading and the content beneath it (down to the next heading) form ONE block.- A field label and its value are ONE block - EVEN in a dense report header. Write "Visit Date: 08/12/2025" as one record; NEVER split the label from its value ("Visit Date:" + "08/12/2025" as two records is WRONG).- On forms, when a key has an associated value, merge the key and value into the same block and bbox so the association is explicit. If several related key-value fields are visually grouped, emit the group as one block with one "Label: value" line per field.- A bordered or visually-grouped PANEL (e.g. a "PRESCRIBER INFORMATION" box with all its fields) is ONE block - join its label:value pairs with "\n".- A row of related cells that is NOT a clean table (a lab-result line, a transaction line) is ONE block - join the cells into one line; NEVER one record per cell.- NEVER emit a bare value, a single cell, or a lone field as its own record.- Keep COLUMNS separate: two side-by-side panels (e.g. Patient | Ordering Provider, Bill To | Ship To) are TWO blocks - never merge across the gutter.- DO NOT merge unrelated neighboring panels or sections.- Group by semantic relationship, NOT by bbox size. Small unrelated regions must stay separate. Example: a page title at the top-left and a page number at the top-right are TWO blocks, even if both boxes are small and on the same horizontal band. Only group items that belong to the same section, panel, list, table, or key-value group.- Prefer CORRECT grouping over tight boxes: a block's bbox may be wide and may lightly touch a neighbor - do NOT over-split a section just to keep boxes small or separate.- Page-edge content: if visible text touches or sits near the page boundary, inspect the full edge carefully and make the bbox include the entire visible glyphs/block, even if the box must start at 0 or end at 1000. Do NOT shrink edge boxes inward.- Bbox coverage is strict: every transcribed character in text_content MUST be inside that record's bbox_2d, with no clipped letters. This matters most for small regions, rotated/non-horizontal text, page-edge text, headers/footers, stamps, and fax strips. Use a tight box around the actual region, but never make it so tight that any visible character you transcribed falls outside the box.Output JSON only.Checkboxes: transcribe every checkbox as [x] if marked or [ ] if unmarked, placed before its label (e.g. "[x] Allergies reviewed"; Y/N pairs as "Y [x] N [ ]"). Include every checkbox, including checkbox grids and Y/N option pairs.{ "type": "array", "items": { "type": "object", "properties": { "bbox_2d": { "type": "array", "items": {"type": "integer"}, "minItems": 4, "maxItems": 4 }, "text_content": {"type": "string"} }, "required": ["bbox_2d", "text_content"] }}import torchfrom PIL import Imagefrom transformers import AutoModelForImageTextToText, AutoProcessor MODEL = "hanji-dev/hanji-parse-4b"model = AutoModelForImageTextToText.from_pretrained( MODEL, dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(MODEL) image = preprocess(Image.open("page.png")) # see preprocessing abovePROMPT = "..." # the exact prompt above messages = [{ "role": "user", "content": [ {"type": "image", "image": image}, {"type": "text", "text": PROMPT}, ],}]inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",).to(model.device) out = model.generate(**inputs, max_new_tokens=8192, do_sample=False)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))python -m sglang.launch_server \ --model-path hanji-dev/hanji-parse-4b \ --attention-backend fa3 \ --mm-attention-backend fa3 \ --mem-fraction-static 0.85 \ --chunked-prefill-size 8192 \ --speculative-algorithm NGRAM \ --speculative-num-draft-tokens 16 \ --speculative-ngram-max-bfs-breadth 10 \ --enable-deterministic-inference \ --context-length 16384