About

This repository contains the VideoKR-Qwen3-VL-8B-SFT model presented in VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding (ICML 2026 Spotlight).

VideoKR-Qwen3-VL-8B-SFT is obtained by supervised fine-tuning Qwen3-VL-8B-Instruct on VideoKR-SFT-201K for one epoch. Each training example includes a high-quality chain-of-thought (CoT) rationale as the supervision target. This SFT checkpoint serves as the starting point for subsequent GRPO reinforcement learning, yielding the final VideoKR-Qwen3-VL-8B model.

Links

Performance

Results with 128 input frames:

The SFT checkpoint already shows strong gains on knowledge-intensive benchmarks (e.g., +4.6 on VideoKR-Eval, +3.4 on MMVU) compared to the base model, while the subsequent GRPO stage further recovers general video reasoning performance.

Training

For detailed training instructions, please refer to the GitHub repository.

bash
Copy code
cd /path/to/VideoKR/llamafactory
conda activate videokr_train

# Prepare SFT data
mkdir -p data/raw
huggingface-cli download minuzero/VideoKR-Train \
  --repo-type dataset --local-dir data/raw \
  --include "VideoKR-COT-201K.jsonl"

python local_script/prepare_videokr_sft_data.py \
  --input data/raw/VideoKR-COT-201K.jsonl \
  --output data/videokr_train.json

# Launch SFT
bash local_script/train_videokr.sh qwen3vl

Citation

If you find VideoKR useful in your research, please cite our paper:

bibtex
Copy code
@misc{fu2026videokrknowledgereasoningintensivevideo,
      title={VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding}, 
      author={Lin Fu and Zheyuan Yang and Yang Wang and Tingyu Song and Arman Cohan and Yilun Zhao},
      year={2026},
      eprint={2606.05259},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2606.05259}, 
}