Introduction
UI-TARS-1.5, an open-source multimodal agent built upon a powerful vision-language model. It is capable of effectively performing diverse tasks within virtual worlds.
Leveraging the foundational architecture introduced in our recent paper, UI-TARS-1.5 integrates advanced reasoning enabled by reinforcement learning. This allows the model to reason through its thoughts before taking action, significantly enhancing its performance and adaptability, particularly in inference-time scaling. Our new 1.5 version achieves state-of-the-art results across a variety of standard benchmarks, demonstrating strong reasoning capabilities and notable improvements over prior models.
Code: https://github.com/bytedance/UI-TARS
Application: https://github.com/bytedance/UI-TARS-desktop
Online Benchmark Evaluation
Table with columns: Benchmark type, Benchmark, UI-TARS-1.5, OpenAI CUA, Claude 3.7, Previous SOTA| Benchmark type | Benchmark | UI-TARS-1.5 | OpenAI CUA | Claude 3.7 | Previous SOTA |
|---|
| Computer Use | OSworld (100 steps) | 42.5 | 36.4 | 28 | 38.1 (200 step) |
| Windows Agent Arena (50 steps) | 42.1 | - | - | 29.8 |
| Browser Use | WebVoyager | 84.8 | 87 | 84.1 |
Grounding Capability Evaluation
Table with columns: Benchmark, UI-TARS-1.5, OpenAI CUA, Claude 3.7, Previous SOTA| Benchmark | UI-TARS-1.5 | OpenAI CUA | Claude 3.7 | Previous SOTA |
|---|
| ScreensSpot-V2 | 94.2 | 87.9 | 87.6 | 91.6 |
| ScreenSpotPro | 61.6 | 23.4 | 27.7 | 43.6 |
Poki Game
Table with columns: Model, 2048, cubinko, energy, free-the-key, Gem-11, hex-frvr, Infinity-Loop, Maze:Path-of-Light, shapes, snake-solver, wood-blocks-3d, yarn-untangle, laser-maze-puzzle, tiles-master| Model | 2048 | cubinko | energy | free-the-key | Gem-11 | hex-frvr | Infinity-Loop |
|---|
Minecraft
Table with columns: Task Type, Task Name, VPT, DreamerV3, Previous SOTA, UI-TARS-1.5 w/o Thought, UI-TARS-1.5 w/ Thought| Task Type | Task Name | VPT | DreamerV3 | Previous SOTA | UI-TARS-1.5 w/o Thought | UI-TARS-1.5 w/ Thought |
|---|
| Mine Blocks | (oak_log) | 0.8 | 1.0 | 1.0 | 1.0 | 1.0 |
| (obsidian) | 0.0 |
Model Scale Comparison
This table compares performance across different model scales of UI-TARS on the OSworld benchmark.
Table with columns: Benchmark Type, Benchmark, UI-TARS-72B-DPO, UI-TARS-1.5-7B, UI-TARS-1.5| Benchmark Type | Benchmark | UI-TARS-72B-DPO | UI-TARS-1.5-7B | UI-TARS-1.5 |
|---|
| Computer Use | OSWorld | 24.6 | 27.5 | 42.5 |
| GUI Grounding | ScreenSpotPro | 38.1 | 49.6 | 61.6 |
The released UI-TARS-1.5-7B focuses primarily on enhancing general computer use capabilities and is not specifically optimized for game-based scenarios, where the UI-TARS-1.5 still holds a significant advantage.
What's next
We are providing early research access to our top-performing UI-TARS-1.5 model to facilitate collaborative research. Interested researchers can contact us at TARS@bytedance.com.
Citation
If you find our paper and model useful in your research, feel free to give us a cite.
@article{qin2025ui,
title={UI-TARS: Pioneering Automated GUI Interaction with Native Agents},
author={Qin, Yujia and Ye, Yining and Fang, Junjie and Wang, Haoming and Liang, Shihao and Tian, Shizuo and Zhang, Junda and Li, Jiahao and Li, Yunxin and Huang, Shijue and others},
journal={arXiv preprint arXiv:2501.12326},
year={2025}
}
