yuanqianhao

Vision-OPD-9B

README

License: apache-2.0

Model Summary

Vision-OPD-9B is a multimodal LLM built on Qwen/Qwen3.5-9B and trained with Vision-OPD (Vision On-Policy Distillation), the regional-to-global self-distillation framework introduced in Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation.

MLLMs often answer fine-grained questions correctly when given an evidence-centered crop, yet fail on the corresponding full image — a persistent regional-to-global perception gap. Vision-OPD closes this gap by letting the model act as its own teacher: a crop-conditioned teacher and a full-image-conditioned student are instantiated from the same MLLM, the student generates on-policy rollouts, and training minimizes the token-level divergence between the two distributions along these rollouts. The benefit of visual zooming is thereby internalized into a single forward pass — no external teacher, no ground-truth labels, no reward verifiers, and no inference-time tool use.

With only 6.2K synthetic samples, Vision-OPD-9B outperforms much larger open-source models (e.g., Qwen3.5-397B, Kimi-K2.6), closed-source models (e.g., GPT-5.4, Gemini-3.1-Pro), and "Thinking-with-Images" agentic methods on fine-grained visual understanding benchmarks, while preserving general multimodal abilities on holdout tasks.

Key Features

🔍 Sees fine details in one forward pass — no cropping tools, zoom-in calls, or multi-step agentic inference at test time.
🪞 Self-distillation, label-free — the model's own crop-conditioned perception supervises its full-image policy; no external teacher, ground-truth labels, or reward verifiers.
🎯 On-policy + dense supervision — token-level divergence (JSD, β = 0.5) on the student's own rollouts avoids the exposure bias of SFT and the sparse rewards of RLVR.
📉 Data-efficient — trained on only 6.2K automatically synthesized samples (Vision-OPD-6K), one epoch.
🧠 No forgetting — general multimodal capability is preserved on holdout tasks (MMVP, CV-Bench, MMStar, POPE).

How It Works

Data synthesis — evidence regions are proposed on unlabeled images via object identification and segmentation; a question answerable from the crop alone is generated, and the region's bounding box is overlaid on the full image for grounding. This yields triplets (full image, crop, question).
Two policies, one model — the teacher conditions on the privileged crop, the student on the full image.
On-policy distillation — the student samples a rollout (up to 1024 tokens); at every position both policies are evaluated on the same student prefix, and the per-token JSD between them is minimized (top-K = 100 logits distillation). Gradients flow only through the student.
EMA teacher regularization — the teacher is updated as an exponential moving average of the student (α = 0.05), which prevents teacher–student co-adaptation and training collapse.

Quickstart

Weights load with standard transformers (≥ 5.5):

python
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "yuanqianhao/Vision-OPD-9B"
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"},
        {"type": "text", "text": "What is the color of the label on the bottle in the background?"},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Or serve with vLLM (OpenAI-compatible API), as used in our evaluation:

bash
vllm serve yuanqianhao/Vision-OPD-9B --port 8000

To reproduce the reported numbers, use the released evaluation harness:

bash
git clone https://github.com/VisionOPD/Vision-OPD && cd Vision-OPD
bash eval/run_eval.sh   # see README for API base / judge / benchmark env vars

Training Details

Table

Base model	`Qwen/Qwen3.5-9B` (non-thinking mode)
Data	Vision-OPD-6K — 6.2K synthesized (full image, crop, question) triplets
Objective	Token-level JSD (β = 0.5) on on-policy student rollouts, top-K = 100 logits distillation
Teacher	Same model conditioned on the privileged crop, EMA-regularized (α = 0.05)
Rollout length	1024 tokens, 1 epoch

Training code: github.com/VisionOPD/Vision-OPD.

Citation

bibtex
@article{yuan2026vision,
  title={Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation},
  author={Yuan, Qianhao and Lou, Jie and Yu, Xing and Lin, Hongyu and Sun, Le and Han, Xianpei and Lu, Yaojie},
  journal={arXiv preprint arXiv:2605.18740},
  year={2026}
}

License

Released under Apache-2.0, following the base model Qwen/Qwen3.5-9B.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

yuanqianhao

Model Tree

Base

Qwen/Qwen3.5-9B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality