Benchmark results
Table with columns: Metric, Pearson r with human ratings| Metric | Pearson r with human ratings |
|---|
| PA | 0.645 |
| IA | 0.775 |
| Overall | 0.719 |
The exact released predictions reproduce PA 0.644617, IA 0.775028, and pooled
Overall 0.719211 with the repository evaluator.
Download and inference
git clone https://github.com/SiyuanMaCS/robojudge-iclr2027.gitcd robojudge-iclr2027pip install -r requirements.txt hf download HuggingFriends/RoboJudge-9B \ --local-dir checkpoints_hf/RoboJudge-9B CKPT=checkpoints_hf/RoboJudge-9B \DATA_ROOT=/path/to/mllm-as-embodied-world-judge \OUT=outputs/robojudge_predictions.jsonl \bash code/run_inference.sh
A smaller BF16 sharded copy and the exact one-epoch SFT checkpoint are available
at https://huggingface.co/HuggingFriends/robojudge-iclr2027-checkpoints.
Checkpoint identity
- Weight format: safetensors
- Original training dtype: FP32
model.safetensors SHA256:
34e17912b402182ab8588494d2715461c7120761e98f8b055b1adbbee400e103
Data
License
The RoboJudge release code and model weights are provided under Apache-2.0.
Use of the associated datasets is additionally subject to the terms of their
upstream sources.