Tasks and output
Use the exact prompts in ocr_prompts.json:
Table with columns: Prompt key, Input and output| Prompt key | Input and output |
|---|
T1_physical | Cropped text region → text, language, script |
T2_physical, T2_reflow | Page transcription, with the selected line-handling convention |
T3_line, T3_block | Ordered text regions with boxes, plus language and script |
T4_line, T4_block | Ordered text-region boxes only |
Region boxes use bbox_2d: [x1, y1, x2, y2], with integer corners on a 0–1000 grid relative to the supplied image and origin at its top-left. They are not pixel [x, y, width, height] boxes. The prompts request exact transcription without translation or spelling correction.
vLLM serving recipe
The standalone FP8 model completed evaluation on 500 test examples using the local vLLM 0.32.0 server with Transformers 5.19.0. All 500 generations completed with schema-valid JSON, with thinking disabled. Schema validity does not mean every transcription is correct; accuracy results are below. No serving-speed or VRAM parity with BF16 is claimed.
vllm serve shbaydadaev/Qwen3.8-27B-ocr-chg_fa-fp8 \ --served-model-name ocr-ab \ --dtype bfloat16 --quantization fp8 \ --max-model-len 16384 --max-num-seqs 1 \ --limit-mm-per-prompt '{"image":1,"video":0}' \ --mm-processor-kwargs '{"max_pixels":4194304}' \ --default-chat-template-kwargs '{"enable_thinking":false}' \ --gpu-memory-utilization 0.80 \ --host 127.0.0.1 --port 8000
The server controls the image budget through --mm-processor-kwargs. Keep the request free of mm_processor_kwargs: vLLM 0.32.0 rejects per-request multimodal overrides by default. The client below uses the server's 4,194,304-pixel setting.
After the server is ready, send one image and the matching task prompt to its chat-completions endpoint. This example normalizes image orientation, composites transparency onto white, and sends PNG pixels:
import base64import ioimport jsonfrom urllib.request import Request, urlopenfrom PIL import Image, ImageOpsfrom huggingface_hub import hf_hub_download repo = "shbaydadaev/Qwen3.8-27B-ocr-chg_fa-fp8"with open(hf_hub_download(repo, "ocr_prompts.json"), encoding="utf-8") as f: prompts = json.load(f)with Image.open("page.png") as source: rgba = ImageOps.exif_transpose(source).convert("RGBA") image = Image.alpha_composite( Image.new("RGBA", rgba.size, "white"), rgba, ).convert("RGB")buffer = io.BytesIO()image.save(buffer, format="PNG")encoded = base64.b64encode(buffer.getvalue()).decode("ascii")payload = { "model": "ocr-ab", "messages": [{"role": "user", "content": [ {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{encoded}"}}, {"type": "text", "text": prompts["T2_physical"]}, ]}], "temperature": 0, "max_tokens": 8192, "chat_template_kwargs": {"enable_thinking": False},}request = Request( "http://127.0.0.1:8000/v1/chat/completions", data=json.dumps(payload).encode("utf-8"), headers={"Content-Type": "application/json"}, method="POST",)with urlopen(request, timeout=900) as response: result = json.load(response)choice = result["choices"][0]if choice["finish_reason"] != "stop": raise RuntimeError(f"Incomplete generation: {choice['finish_reason']}")print(json.dumps(json.loads(choice["message"]["content"]), ensure_ascii=False, indent=2))
Thinking is disabled and decoding is deterministic. Parse and validate the task-specific JSON, and retain incomplete or invalid outputs as failures during evaluation.
Evaluation results and limitations
These results measure this standalone FP8 export, served through vLLM. All 500/500 responses completed with schema-valid JSON; there were no request errors or incomplete generations.
The evaluation used the same fixed first 500 of 3,743 prepared Test records (13.36%) as the merged BF16 comparison: 434 page transcriptions (T2) and 66 cropped-region transcriptions (T1). This prefix is not a balanced or random benchmark, and it does not cover the full test set. Raw corpus character error rate (CER) and word error rate (WER) use prepared source labels, which have not undergone an independent transcription audit.
Table with columns: FP8 evaluation group, Examples, Raw CER, Raw WER| FP8 evaluation group | Examples | Raw CER | Raw WER |
|---|
| All selected examples | 500 | 4.63% | 14.69% |
Chagatai (chg) | 105 | 37.43% | 79.66% |
Persian (fa) | 381 | 2.34% | 10.47% |
Mixed language (mixed) | 13 | 30.58% | 76.55% |
Arabic (ar) | 1 | 33.33% | 100.00% |
The aggregate is heavily weighted toward Persian/Wikisource text; it does not describe Chagatai accuracy. The Arabic result contains only one example and cannot characterize Arabic OCR performance. Schema-valid JSON can still contain transcription errors.
See eval_summary.json for the release summary and evaluation/fp8_test_500.json for the complete sanitized aggregate report, including source, task, language, normalization, and failure-policy details.
Comparison with merged BF16
On the same 500 records, the companion merged BF16 model recorded 4.90% CER / 15.09% WER, compared with 4.63% / 14.69% for FP8. BF16 completed 499 responses; one response reached the generation limit and was scored as an empty transcription under the evaluation failure policy. FP8 completed that response. This one recovered response accounts for 95.57% of the net reduction in corpus character edits, so the aggregate improvement largely reflects recovery from that incomplete generation.
Chagatai CER increased slightly, from 37.32% with merged BF16 to 37.43% with FP8; Chagatai WER changed from 80.18% to 79.66%. These results do not show a uniform improvement across languages or metrics.
FP8 ran through vLLM, while merged BF16 ran through Transformers. The selected records and declared generation settings matched, but the inference backend also changed. This comparison therefore does not isolate quantization effects, establish accuracy parity, or establish serving-speed or VRAM parity. The earlier unmerged BF16-plus-LoRA adapter evaluation is a separate measurement and is not the source of the FP8 results above.
T3/T4 page-grounding training uses synthetic pages only, with no completed held-out real-page grounding benchmark. Training used the original supplied synthetic images rather than regenerated corrected renderings. This transcription evaluation does not measure real-page localization accuracy.
License and provenance
See LICENSE for the upstream Apache-2.0 terms. Source, merge, and quantization provenance are included with machine-local paths removed. SHA256SUMS covers the published files. No training images are included.