Uni-MOPD
Strong Starts, Simple Recipes: Unified Multi-Teacher On-Policy Distillation
from Multimodal Specialists
Multi-teacher on-policy distillation (MOPD) integrates separately trained domain
specialists into a unified model: the student generates responses to training
queries, and the teacher of each query's domain supervises them. Existing
methods refine the distillation procedure while the student starts from a base
model or a multi-domain supervised fine-tuning checkpoint, which leaves
capability integration largely to distillation itself.
Unified MOPD (Uni-MOPD) treats the starting point as a design variable and
pairs a strong start with a simple recipe: a training-free uniform merge
of specialists from the same base model initializes the student, and short MOPD
with uniform domain sampling and full-vocabulary reverse KL then refines it.
The bank covers five domains: STEM reasoning, general visual question answering,
document/OCR/chart understanding, GUI grounding, and embodied understanding. All
five specialists are post-trained from the same base checkpoint,
Qwen/Qwen3-VL-30B-A3B-Instruct, an instruction-tuned
mixture-of-experts vision-language model, and follow different post-training
paths, each ending
with reinforcement learning on its own domain. Because they share the base
model, the specialists share one architecture and vocabulary, which permits both
weight-space merging and token-level distillation.
Table with columns: Repository, What it is| Repository | What it is |
|---|
Uni-MOPD | The unified student |
Uni-MOPD-Merge | The uniform merge that initializes the student |
Uni-MOPD-Teacher-STEM | STEM reasoning specialist |
Uni-MOPD-Teacher-VQA | General visual question answering specialist |
Uni-MOPD-Teacher-OCR | Document/OCR/chart understanding specialist (RL from a multi-domain SFT start) |
Uni-MOPD-Teacher-OCR-Bank2 | Document/OCR/chart understanding specialist (OCR-domain SFT and RL) |
Uni-MOPD-Teacher-GUI | GUI grounding specialist |
Uni-MOPD-Teacher-Embodied | Embodied understanding specialist |
Uni-MOPD-Teacher-OCR-Bank2 comes from the paper's second specialist bank.
The two OCR specialists follow different post-training paths: the first
starts its domain reinforcement learning from a multi-domain SFT checkpoint,
which gives it broader coverage, while the second is trained with supervised
fine-tuning and reinforcement learning on the document/OCR domain only.
All of them sit under the AmberHeart namespace; Uni-MOPD-Teacher-OCR is this
repository.
This model
One of the five specialists in the bank. It is post-trained
from the shared base checkpoint along its own domain's path, ending with
reinforcement learning on that domain, and it leads its own domain among the
five.
Weights are bfloat16, in 14 safetensors shards.
Usage
from transformers import AutoProcessor, Qwen3VLMoeForConditionalGeneration model_id = "AmberHeart/Uni-MOPD-Teacher-OCR"model = Qwen3VLMoeForConditionalGeneration.from_pretrained( model_id, dtype="auto", device_map="auto")processor = AutoProcessor.from_pretrained(model_id) messages = [{ "role": "user", "content": [ {"type": "image", "url": "<your-image>"}, {"type": "text", "text": "Describe this image."}, ],}]inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",).to(model.device)out = model.generate(**inputs, max_new_tokens=128)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Requires transformers>=4.57.1.
License
Apache License 2.0, inherited from the base model. See LICENSE for the license
text and NOTICE for the statement of modifications.