Evaluation
On AndroidWorld (116 tasks), this checkpoint obtains 78/116 (67.2%) Pass@1, 87/116 (75.0%) Pass@2, and 90/116 (77.6%) Pass@3. On the MobileWorld GUI-only split (117 tasks), it obtains 48/117 (41.0%) success rate.
Reproduction code and raw evaluation archives are available from the anonymous code repository and benchmark-results dataset.
Usage
Use the same loading and prompting interface as mPLUG/GUI-Owl-1.5-8B-Instruct. See the anonymous code repository for AndroidWorld and MobileWorld runners.
Limitations
The model can make incorrect or unsafe GUI actions. Run it only in isolated test environments, inspect actions before using it with personal data, and do not treat benchmark success as evidence of general reliability.