How to run inference
pip install -r inference/requirements.txt python inference/infer_mdpbench.py \ --model . \ --image-dir /path/to/MDPBench_img_public \ --out-dir /path/to/result/our_model
Writes one .md per page image (filename = image stem) — exactly what the MDPBench
evaluation pipeline expects. See SUBMISSION_README.md for the full recipe and the exact
inference settings, and VALIDATION.md for what was tested on real hardware.
Key settings that must be kept: official MDPBench prompt (embedded verbatim in
inference/infer_mdpbench.py), greedy decoding, max_new_tokens=8192, native resolution
(max_pixels=16777216), enable_thinking=False, and the Conv3d → Linear patch for the
vision patch-embed (applied automatically by the script).
Note: the vision patch-embed is nn.Conv3d in stock transformers; on some GPUs this hits
a pathological cuDNN kernel (~39.5 s per call, 99.8% of the forward pass). The provided
script replaces it with a mathematically equivalent Linear matmul before loading.