GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
PaddlePaddle
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
PaddlePaddle
Model Tree
OpenGVLab/InternVL3_5-1B
Input Modalities
Output Modalities
Supported Functionality
2026.07 🎉 A live demo is available at hugging-apps/hpd-parsing. Thanks to multimodalart for building it!2026.07 🚀 We release HPD-Parsing, a 1B hierarchical parallel document parser. It achieves 94.91% overall on OmniDocBench v1.6 and 4,752 TPS peak throughput.We introduce HPD-Parsing, a lightweight (1B) and high-throughput document parsing model built on a Hierarchical Parallel Decoding paradigm. Unified VLM-based parsers process an entire page jointly but generate the output through a single token-by-token autoregressive trajectory, creating a sequential bottleneck that grows with document length. HPD-Parsing is motivated by a key property of document parsing: page structure requires global coordination, whereas content generation is largely localized within individual regions. Based on this observation, a main layout branch coordinates the global document structure and dynamically dispatches localized content generation to concurrent branches, while Progressive Multi-Token Prediction (P-MTP) further reduces the decoding steps within each branch. HPD-Parsing achieves an overall score of 94.91% on OmniDocBench v1.6 — a new state of the art among end-to-end unified parsers — while reaching a peak throughput of 4,752 TPS, 2.62× the fastest existing document parser and 3.06× its own autoregressive baseline.
🚀 Hierarchical Parallel Decoding for High-Throughput Document Parsing: We introduce Hierarchical Parallel Decoding (HPD), a new decoding paradigm that restructures full-page autoregressive generation into globally coordinated, localized parallel decoding. A main layout branch performs global coordination and dynamically decomposes the conventional single decoding trajectory into concurrent content branches, each responsible for a localized document region. Within each branch, P-MTP further reduces the number of decoding steps by predicting multiple future tokens at each iteration. Together with shared-prefix KV cache reuse, HPD substantially shortens the effective sequential decoding path along both branch and token dimensions.
🔄 Staged Adaptation with Automated Difficulty-Aware Data Curation: We develop a staged adaptation strategy that transfers conventional autoregressive document parsing capabilities to the proposed hierarchical parallel decoding paradigm while preserving parsing accuracy. The strategy is supported by an automated difficulty-aware data curation pipeline that integrates large-scale data collection, model-assisted annotation, difficulty estimation, and balanced sampling. By progressively adapting the model and emphasizing challenging samples, the training framework mitigates the accuracy degradation caused by the transition to parallel decoding with minimal manual annotation effort.
⚡ State-of-the-Art Throughput with Competitive Parsing Accuracy: HPD-Parsing achieves state-of-the-art inference efficiency on OmniDocBench v1.6, reaching a peak throughput of 4,752 Tokens Per Second (TPS). It delivers 1.62× the throughput of the fastest existing document parsing model and more than 3.06× that of its autoregressive baseline, while maintaining competitive parsing accuracy. These results demonstrate that document parsing can be effectively executed through global layout coordination and localized parallel decoding rather than a single sequential generation trajectory.
HPD-Parsing adopts InternVL3.5-1B as its backbone, applies dynamic tile-based cropping (up to 24 tiles of 448×448) to preserve high-resolution details. Its primary departure from conventional unified parsers is the decoding paradigm: instead of generating the entire page along a single autoregressive trajectory, HPD-Parsing employs a main layout branch to coordinate the global structure and spawns localized content branches for concurrent region-level decoding, with P-MTP integrated into each branch.
HPD-Parsing runs on a customized build of vLLM (based on vLLM v0.17.1) that implements the dynamic request forking required by hierarchical parallel decoding and adapts P-MTP speculative decoding.
The Docker image ships the customized vLLM build and all dependencies, which is the easiest way to get started. Its default entrypoint runs the Python API example below out of the box.
With the online Docker image, start the container directly. It downloads the model and starts the inference server, which listens on port 8118 by default:
shell
docker run \-it \--rm \--gpus all \--network host \ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/hpd-parsing-vllm:latest-nvidia-gpu
Without Docker, install the customized vLLM prebuilt package (Python 3.10–3.13, NVIDIA driver with CUDA 12.8+) in a virtual environment to avoid dependency conflicts:
shell
python -m venv .venv_hpd_parsingsource .venv_hpd_parsing/bin/activatepython -m pip install https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/deploy/hpd_parsing/vllm-0.17.1+hpdparsing-cp38-abi3-manylinux_2_31_x86_64.whl
Then run inference with the vLLM Python API:
python
# Set the environment variable before running: export MAX_PATCHES_WITH_RESIZE=trueimport base64from vllm import LLM, SamplingParamsllm = LLM(model="PaddlePaddle/HPD-Parsing",trust_remote_code=True,max_model_len=16384,limit_mm_per_prompt={"image": 1},gpu_memory_utilization=0.9,attention_backend="FLASHINFER",enable_prefix_caching=True,speculative_config={"method": "medusa","model": "PaddlePaddle/HPD-Parsing/P-MTP","num_speculative_tokens": 6,},)sampling_params = SamplingParams(temperature=0, max_tokens=8000)with open("demo.png", "rb") as f:image_base64 = base64.b64encode(f.read()).decode("utf-8")messages = [{"role": "user","content": [{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_base64}"}},{"type": "text", "text": "document parsing with fork."},],}]outputs = llm.chat(messages=messages, sampling_params=sampling_params)print(outputs[0].outputs[0].text)
model.generate_hpd(...) reproduces the vLLM decoding paradigm in plain transformers (single image, batch size 1): the parent layout branch is decoded greedily and every <FORK> token spawns a content child branch that inherits the parent KV cache (shared-prefix reuse); the branches are then spliced back into one sequence. The P-MTP/ head is used for speculative decoding when use_mtp=True.
The P-MTP weights ship inside the main checkpoint (keys language_model.mtp.*) and are loaded automatically by from_pretrained; load_mtp_weights() simply enables them. To load them from the standalone P-MTP/ directory instead, pass its path.
Image preprocessing (dynamic tiling that mirrors vLLM's InternVL path with MAX_PATCHES_WITH_RESIZE=true) lives in image_preprocess.py; load_image returns the stacked pixel_values tensor consumed by generate_hpd.
python
import torchfrom transformers import AutoModel, AutoTokenizerfrom image_preprocess import load_imageMODEL = "PaddlePaddle/HPD-Parsing"DEVICE = "cuda" if torch.cuda.is_available() else "cpu"model = AutoModel.from_pretrained(MODEL, torch_dtype=torch.bfloat16, trust_remote_code=True,).eval().to(DEVICE)tokenizer = AutoTokenizer.from_pretrained(MODEL, trust_remote_code=True, use_fast=False)model.load_mtp_weights() # or model.load_mtp_weights(f"{MODEL}/P-MTP")pixel_values = load_image("test.png").to(torch.bfloat16).to(DEVICE)response = model.generate_hpd(tokenizer,pixel_values,"document parsing with fork.", # `document parsing.` for non-fork layoutdict(max_new_tokens=8000),use_mtp=True, # P-MTP speculative decoding (parent branch)num_speculative_tokens=6,batch_children=False, # see note below)print(response)
Key arguments:
use_mtp (default False) / num_speculative_tokens (default 6): enable P-MTP speculative decoding with the given draft length; requires load_mtp_weights(...). Greedy verification keeps output identical to plain autoregressive decoding.batch_children (default False): serial per-child decoding that reuses the parent KV cache in place — lowest memory and fastest in practice. Set True to decode all children concurrently in one left-padded batch (same output, but heavier since transformers duplicates the shared prefix KV per child).return_token_ids (default False): also return the final token id list.Reference implementation only. Use the vLLM path for production throughput — its paged KV cache enables true concurrent branches with zero-copy prefix sharing.
With only 1B parameters, HPD-Parsing establishes a new state of the art among end-to-end unified parsers. Best results in each column are in bold.
Throughput under batch size 512 on NVIDIA A800 80GB with vLLM, HPD-Parsing increases throughput from 1.02 to 2.68 PPS and from 1,554.8 to 4,752.1 TPS, corresponding to improvements 2.62× and 3.06×, respectively. Despite processing approximately 4,800 input tokens per page, over four times that of DeepSeek-OCR-2, HPD-Parsing still achieves 1.31× higher PPS and 1.62× higher TPS, demonstrating strong inference efficiency under a considerably larger input-token budget.Its acceleration advantage grows with document length, reaching up to 18.04× fewer decoding steps, 3.67× higher request throughput, and 5.80× lower single-request latency in the longest output-length bucket.
Scripts under eval/ reproduce both the throughput (TPS) and the OmniDocBench v1.6 accuracy numbers via a decoupled infer -> convert -> evaluate pipeline:
eval/benchmark_tps.py — batched vLLM inference that reports TPS metrics and dumps the raw predictions in one run.eval/hpd_to_markdown.py — converts the <BLOCK>...<CHILD>... predictions into per-page markdown for OmniDocBench's end2end evaluation.See eval/README.md for the full workflow, including how to run the official OmniDocBench evaluation on the generated markdown.
We would like to thank InternVL and Qwen3 for the backbone, vLLM for the serving framework, and PaddleOCR-VL, MinerU and OmniDocBench for providing valuable data, model weights and benchmarks. We also appreciate everyone's contribution to this open-source project!
If you find HPD-Parsing helpful, feel free to give us a star and citation.
bibtex
@misc{wei2026hpdparsinghierarchicalparalleldocument,title={HPD-Parsing: Hierarchical Parallel Document Parsing},author={Shu Wei and Jingjing Wu and Lingshu Zhang and Qunyi Xie and Hao Zou and Le Xiang and Xu Fan and Yangliu Xu and Manhui Lin and Xiaolong Ma and Cheng Cui and Tengyu Du and YY},year={2026},eprint={2607.18839},archivePrefix={arXiv},primaryClass={cs.CL},url={https://arxiv.org/abs/2607.18839},}