GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Gyubeum
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Gyubeum
Model Tree
Qwen/Qwen3-VL-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
The model was trained on a multi-turn format:
text
[system] You are a helpful assistant. + mobile_use tool spec (with screen resolution)[user] The user query: <goal> Task progress (You have done the following operation on the current device): Step1: <action> <image> ... StepN: <action>; <image>[assistant] <tool_call>{"name": "mobile_use", "arguments": {...}}</tool_call>[user] Was the agent's action above correct given the current screen state? Answer with exactly one token: <|+|> for correct, <|-|> for wrong.[assistant] ← model generates <|+|> or <|-|>Use this model as a process reward model (PRM) to evaluate GUI agent actions on mobile UIs. It is well-suited for:
bash
pip install vllm transformers torch pillowpython
from vllm import LLM, SamplingParamsfrom transformers import AutoProcessorfrom PIL import Image MODEL_PATH = "Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie" POS_TOKEN = "<|+|>"NEG_TOKEN = "<|-|>" JUDGE_PROMPT = ( "Was the agent's action above correct given the current screen state? " "Answer with exactly one token: <|+|> for correct, <|-|> for wrong.") SYSTEM_PROMPT = ( "You are a helpful assistant.\n\n# Tools\n\n" "You may call one or more functions to assist with the user query.\n\n" "You are provided with function signatures within <tools></tools> XML tags:\n" "<tools>\n" '{"type": "function", "function": {"name": "mobile_use", ' '"description": "Use a touchscreen to interact with a mobile device. ' "The screen's resolution is 540x1200.\", " '"parameters": {"properties": {"action": {"type": "string", ' '"enum": ["click", "long_press", "swipe", "type", "key", "system_button", "open", "wait", "terminate"]}, ' '"coordinate": {"type": "array"}, "text": {"type": "string"}, "button": {"type": "string"}}, ' '"required": ["action"]}}}\n</tools>\n\n' "For each function call, return a json object within <tool_call></tool_call> XML tags:\n" "<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>") # Load the merged model directly — no LoRA adapter needed.llm = LLM( model=MODEL_PATH, dtype="bfloat16", gpu_memory_utilization=0.7, max_model_len=8192, limit_mm_per_prompt={"image": 10}, enforce_eager=True,)processor = AutoProcessor.from_pretrained(MODEL_PATH, max_pixels=1_048_576)tokenizer = processor.tokenizer pos_id = tokenizer.encode(POS_TOKEN, add_special_tokens=False)[0]neg_id = tokenizer.encode(NEG_TOKEN, add_special_tokens=False)[0] sampling_params = SamplingParams(max_tokens=1, temperature=0.0) def score_action(goal, prior_steps_text, tool_call_response, screenshot): """ Score a GUI agent action. Args: goal: Task goal string. prior_steps_text: String like "Step1: tap search\nStep2: type query\n" tool_call_response: The agent's raw tool_call response string to judge. screenshot: PIL.Image of the current screen state. Returns: float: 1.0 (correct), 0.0 (wrong), or 0.5 (undecided). """ user_content = [ {"type": "text", "text": f"The user query: {goal}\nTask progress (...): {prior_steps_text}; "}, {"type": "image"}, ] messages = [ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": user_content}, {"role": "assistant", "content": tool_call_response.strip()}, {"role": "user", "content": JUDGE_PROMPT}, ] prefix_text = processor.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) output = llm.generate( [{"prompt": prefix_text, "multi_modal_data": {"image": [screenshot]}}], sampling_params, )[0] gen_id = output.outputs[0].token_ids[0] if output.outputs[0].token_ids else None if gen_id == pos_id: return 1.0 if gen_id == neg_id: return 0.0 return 0.5 # neither token — treated as undecided # Examplescreenshot = Image.open("screenshot.png").convert("RGB")tool_call = '<tool_call>\n{"name": "mobile_use", "arguments": {"action": "click", "coordinate": [540, 120]}}\n</tool_call>' score = score_action( goal="Tap the search button", prior_steps_text="Step1: opened the app\n", tool_call_response=tool_call, screenshot=screenshot,)print(f"Score: {score}") # 1.0 = correct, 0.0 = wrongrm_evalbash
python eval_rm.py \ --rm_path Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie \ --datasets ui-genie \ --mode discrete \ --uigenie_jsonl /path/to/reward_data_rm_pairs_last5.jsonl \ --uigenie_images_dir /path/to/images \ --output_dir results/Per-side classification accuracy and pair accuracy (chosen classified correct AND rejected classified wrong) on held-out (chosen, rejected) pairs:
| Dataset | Pairs | Pair accuracy | Source |
|---|---|---|---|
| UI-Genie (held-out) | 1000 | 72.0% (720/1000); chosen-side 87.5%, rejected-side 80.1% | Fresh rm_eval run, 2026-08-10, on this exact uploaded checkpoint |
| AndroidFlux (OOD multi-agent replay) | 203 | 19.2% (39/203); chosen-side 29.6%, rejected-side 75.9% | rm_eval run, 2026-04-30, on this exact checkpoint (mtime-verified) |
The AndroidFlux pair accuracy is low because the classifier is biased toward <|-|> on this out-of-domain, multi-agent replay data (chosen-side accuracy 29.6% vs. rejected-side 75.9%) — a known domain-gap limitation of this SFT stage, not an artifact of this upload's fix.
| Field | Value |
|---|---|
| Base model | Qwen/Qwen3-VL-8B-Instruct |
| Training method | SFT (LoRA, merged + trained lm_head baked in) |
| Training data | UI-Genie-RM-517k (64k pairs training split) |
| Output | Discrete preference token: <|+|> / <|-|> |
| Scoring | Greedy-decode 1 token → 1.0 / 0.0 / 0.5 |
bibtex
@misc{qwen3technicalreport, title={Qwen3 Technical Report}, author={Qwen Team}, year={2025}, eprint={2505.09388}, archivePrefix={arXiv}, primaryClass={cs.CL},}[system] You are a helpful assistant. + mobile_use tool spec (with screen resolution)[user] The user query: <goal> Task progress (You have done the following operation on the current device): Step1: <action> <image> ... StepN: <action>; <image>[assistant] <tool_call>{"name": "mobile_use", "arguments": {...}}</tool_call>[user] Was the agent's action above correct given the current screen state? Answer with exactly one token: <|+|> for correct, <|-|> for wrong.[assistant] ← model generates <|+|> or <|-|>from vllm import LLM, SamplingParamsfrom transformers import AutoProcessorfrom PIL import Image MODEL_PATH = "Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie" POS_TOKEN = "<|+|>"NEG_TOKEN = "<|-|>" JUDGE_PROMPT = ( "Was the agent's action above correct given the current screen state? " "Answer with exactly one token: <|+|> for correct, <|-|> for wrong.") SYSTEM_PROMPT = ( "You are a helpful assistant.\n\n# Tools\n\n" "You may call one or more functions to assist with the user query.\n\n" "You are provided with function signatures within <tools></tools> XML tags:\n" "<tools>\n" '{"type": "function", "function": {"name": "mobile_use", ' '"description": "Use a touchscreen to interact with a mobile device. ' "The screen's resolution is 540x1200.\", " '"parameters": {"properties": {"action": {"type": "string", ' '"enum": ["click", "long_press", "swipe", "type", "key", "system_button", "open", "wait", "terminate"]}, ' '"coordinate": {"type": "array"}, "text": {"type": "string"}, "button": {"type": "string"}}, ' '"required": ["action"]}}}\n</tools>\n\n' "For each function call, return a json object within <tool_call></tool_call> XML tags:\n" "<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>") # Load the merged model directly — no LoRA adapter needed.llm = LLM( model=MODEL_PATH, dtype="bfloat16", gpu_memory_utilization=0.7, max_model_len=8192, limit_mm_per_prompt={"image": 10}, enforce_eager=True,)processor = AutoProcessor.from_pretrained(MODEL_PATH, max_pixels=1_048_576)tokenizer = processor.tokenizer pos_id = tokenizer.encode(POS_TOKEN, add_special_tokens=False)[0]neg_id = tokenizer.encode(NEG_TOKEN, add_special_tokens=False)[0] sampling_params = SamplingParams(max_tokens=1, temperature=0.0) def score_action(goal, prior_steps_text, tool_call_response, screenshot): """ Score a GUI agent action. Args: goal: Task goal string. prior_steps_text: String like "Step1: tap search\nStep2: type query\n" tool_call_response: The agent's raw tool_call response string to judge. screenshot: PIL.Image of the current screen state. Returns: float: 1.0 (correct), 0.0 (wrong), or 0.5 (undecided). """ user_content = [ {"type": "text", "text": f"The user query: {goal}\nTask progress (...): {prior_steps_text}; "}, {"type": "image"}, ] messages = [ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": user_content}, {"role": "assistant", "content": tool_call_response.strip()}, {"role": "user", "content": JUDGE_PROMPT}, ] prefix_text = processor.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) output = llm.generate( [{"prompt": prefix_text, "multi_modal_data": {"image": [screenshot]}}], sampling_params, )[0] gen_id = output.outputs[0].token_ids[0] if output.outputs[0].token_ids else None if gen_id == pos_id: return 1.0 if gen_id == neg_id: return 0.0 return 0.5 # neither token — treated as undecided # Examplescreenshot = Image.open("screenshot.png").convert("RGB")tool_call = '<tool_call>\n{"name": "mobile_use", "arguments": {"action": "click", "coordinate": [540, 120]}}\n</tool_call>' score = score_action( goal="Tap the search button", prior_steps_text="Step1: opened the app\n", tool_call_response=tool_call, screenshot=screenshot,)print(f"Score: {score}") # 1.0 = correct, 0.0 = wrongpython eval_rm.py \ --rm_path Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie \ --datasets ui-genie \ --mode discrete \ --uigenie_jsonl /path/to/reward_data_rm_pairs_last5.jsonl \ --uigenie_images_dir /path/to/images \ --output_dir results/@misc{qwen3technicalreport, title={Qwen3 Technical Report}, author={Qwen Team}, year={2025}, eprint={2505.09388}, archivePrefix={arXiv}, primaryClass={cs.CL},}