Intended Use
Scalar reward signal for GUI agent training (PPO/GRPO, RLHF) or best-of-N selection. Unlike the discrete <|+|> / <|-|> classifier, this model outputs a continuous unbounded reward directly usable as a value signal.
Evaluation
Pairwise accuracy on android_flux_recovery_action_preference from Gyubeum/AndroidFlux_RM_Eval — 94 pairs, each two candidate t+1 recovery actions from the same checkpoint, labelled by success under a 12-model continuation panel:
Table with columns: Model, Pairwise accuracy, Ties, Mean margin| Model | Pairwise accuracy | Ties | Mean margin |
|---|
-scoring (unbalanced 64k) | 0.6064 (57/94) | 0 | +1.381 |
| this model (balanced 64k) | 0.4894 (46/94) | 1 | +0.471 |
UI-Genie (discrete SFT) | 0.0106 (1/94) | 90 | −0.021 |
This checkpoint performs at chance on that benchmark. Breaking it out by subset shows no split above chance:
Table with columns: Split, Pairs, Accuracy| Split | Pairs | Accuracy |
|---|
clean_path | 56 | 0.4821 |
error_path | 38 | 0.5000 |
success_rate | 65 | 0.4615 |
combined_success_length | 29 | 0.5517 |
For comparison, the unbalanced -scoring model separates the same pairs with visible structure (0.658 on error_path vs 0.571 on clean_path). The head here does produce well-separated scores — only 1 tie, margins spanning [−17.75, +18.38] — but that separation does not correlate with which action succeeds.
With n=94 the 95% CI is roughly ±10 points, so treat the gap as indicative rather than tight. Evaluated 2026-08-27 with rm_eval --mode bt.
Inference
vLLM's Qwen3VLForConditionalGeneration loader does not support the extra score.weight tensor — use HuggingFace transformers directly.
import json, torch
from safetensors import safe_open
from huggingface_hub import hf_hub_download
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
MODEL_PATH = "Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie-scoring-64k-balanced"
DEVICE = "cuda"
model = Qwen3VLForConditionalGeneration.from_pretrained(
MODEL_PATH, torch_dtype=torch.bfloat16, low_cpu_mem_usage=True,
).to(DEVICE).eval()
processor = AutoProcessor.from_pretrained(MODEL_PATH, max_pixels=1_048_576)
index = json.load(open(hf_hub_download(MODEL_PATH, "model.safetensors.index.json")))
shard = hf_hub_download(MODEL_PATH, index["weight_map"]["score.weight"])
with safe_open(shard, framework="pt") as f:
w = f.get_tensor("score.weight")
score_head = torch.nn.Linear(w.shape[1], w.shape[0], bias=False)
with torch.no_grad():
score_head.weight.copy_(w)
score_head = score_head.to(DEVICE, dtype=torch.bfloat16)
Training Details
Table with columns: Field, Value| Field | Value |
|---|
| Base model | Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie |
| Training method | Bradley-Terry pairwise loss (LoRA, merged) |
| Training data | UI-Genie-RM-517k, class-balanced 64k subset |
| Architecture | Qwen3VLForConditionalGeneration + score.weight linear head |
| Task type | seq_cls (regression, num_labels=1) |
| Score | Last non-padding token hidden state → linear head → scalar |
| dtype |
Citation
@misc{qwen3technicalreport,
title={Qwen3 Technical Report},
author={Qwen Team},
year={2025},
eprint={2505.09388},
archivePrefix={arXiv},
primaryClass={cs.CL},
}
