GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
lime-nlp
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
lime-nlp
Model Tree
Qwen/Qwen3-VL-8B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Attribute | Value |
|---|---|
| Base Model | Qwen3-VL-8B-Instruct |
| Parameters | 8.7B |
| Architecture | Qwen3VLForConditionalGeneration |
| Precision | bfloat16 |
| Max Context Length | 128,000 tokens |
| Video Resolution | 720p (1280x720) |
| Max Video Frames | 50 |
| Video FPS | 1.0 |
| Training Data | OSWorld + AgentNet + ScaleCUA |
| Training Loss | 0.046 |
| Eval Accuracy | 84.7% |
ExeVRM incorporates two token pruning techniques that enable efficient processing of long execution videos:
Combined, STP + TTP achieve 40-60% token reduction while maintaining reward prediction quality.
| Parameter | Value |
|---|---|
use_stp | true |
stp_mode | forward_removal |
stp_threshold | 3.0 |
stp_skip_ratio | 0.0 |
stp_large_comp_threshold |
bash
pip install transformers torch accelerate
For video processing:
bash
pip install av pillow
python
from transformers import Qwen3VLForConditionalGeneration, AutoProcessorimport torchmodel_name = "lime-nlp/ExeVRM-8B" # Replace with the actual HF repo name# Load modelmodel = Qwen3VLForConditionalGeneration.from_pretrained(model_name,torch_dtype=torch.bfloat16,device_map="auto",trust_remote_code=True,)# Load processorprocessor = AutoProcessor.from_pretrained(model_name, trust_remote_code=True)
ExeVRM expects 720p (1280x720) video frames sampled at 1 FPS with a maximum of 50 frames. Here is how to preprocess a video:
python
import avimport numpy as npimport mathdef sample_frames_from_video(video_path, fps=1.0, max_frames=50):"""Sample frames from a video at the specified FPS, up to max_frames.Args:video_path: Path to the video file (mp4, avi, etc.)fps: Target frames per second for sampling (default: 1.0)max_frames: Maximum number of frames to sample (default: 50)Returns:List of PIL Images"""container = av.open(video_path)stream = container.streams.video[0]total_frames = stream.framesduration_seconds = float(stream.duration * stream.time_base)# Compute number of frames to samplesample_frames = max(1, math.floor(duration_seconds * fps))sample_frames = min(total_frames, max_frames, sample_frames)# Uniformly sample frame indicesindices = np.linspace(0, total_frames - 1, sample_frames).astype(np.int32)indices_set = set(indices.tolist())frames = []for i, frame in enumerate(container.decode(video=0)):if i in indices_set:frames.append(frame.to_image()) # Convert to PIL Imageif i > max(indices):breakcontainer.close()return frames
python
from qwen_vl_utils import process_vision_info# Prepare the input messagemessages = [{"role": "user","content": [{"type": "video", "video": "path/to/agent_trajectory.mp4", "fps": 1.0},{"type": "text", "text": ("Given a user task and a computer-using video recording, ""evaluate whether the user completes the task or not. ""Reply your judgement in the \\box{}.\n""If the video correctly completes the task, reply \\box{correct}. ""Otherwise, reply \\box{incorrect}.\n\n""# User Task\n""Open Google Chrome and search for 'weather today'\n")},],}]# Process inputstext = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)image_inputs, video_inputs = process_vision_info(messages)inputs = processor(text=[text],images=image_inputs,videos=video_inputs,padding=True,return_tensors="pt",).to(model.device)# Generatewith torch.no_grad():output_ids = model.generate(**inputs, max_new_tokens=128)# Decodegenerated_ids = output_ids[:, inputs.input_ids.shape[1]:]response = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]print(response)# Expected output: \box{correct} or \box{incorrect}
qwen_vl_utils (Recommended)For the most streamlined experience, install the Qwen VL utilities:
bash
pip install qwen-vl-utils
This provides process_vision_info() which handles video frame extraction and formatting automatically, including frame sampling at the specified FPS.
If you need more control over frame sampling (e.g., for custom preprocessing):
python
# 1. Sample frames manuallyframes = sample_frames_from_video("path/to/video.mp4", fps=1.0, max_frames=50)# 2. Build message with individual frames as imagescontent = [{"type": "video", "video": frames}]content.append({"type": "text","text": ("Given a user task and a computer-using video recording, ""evaluate whether the user completes the task or not. ""Reply your judgement in the \\box{}.\n""If the video correctly completes the task, reply \\box{correct}. ""Otherwise, reply \\box{incorrect}.\n\n""# User Task\n""Your task description here\n"),})messages = [{"role": "user", "content": content}]# 3. Process and run inference (same as above)text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)image_inputs, video_inputs = process_vision_info(messages)inputs = processor(text=[text],images=image_inputs,videos=video_inputs,padding=True,return_tensors="pt",).to(model.device)with torch.no_grad():output_ids = model.generate(**inputs, max_new_tokens=256)generated_ids = output_ids[:, inputs.input_ids.shape[1]:]response = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]print(response)
ExeVRM uses the following prompt template for binary reward prediction:
markdown
Given a user task and a computer-using video recording, evaluate whether the user completes the task or not. Reply your judgement in the \box{}.If the video correctly completes the task, reply \box{correct}. Otherwise, reply \box{incorrect}.# User Task<task description>
If you want the model output justifications, use the following template instead:
markdown
Given a user task and a computer-using video recording, evaluate whether the user completes the task or not. Reply your judgement in the \box{}.If the video correctly completes the task, reply \box{correct}. Otherwise, reply \box{incorrect}.If the video does not complete the task (i.e., incorrect), please provide the timestamp range, i.e., from <[time_start] seconds> to <[time_end] seconds>, of the video that deviates from the user's instruction.# User Task<task description>
\box{correct} / \box{incorrect}) and is not designed for open-ended video QA.If you think this model helps your research, please cite the following paper:
markdown
@misc{song2026videobasedrewardmodelingcomputeruse,title={Video-Based Reward Modeling for Computer-Use Agents},author={Linxin Song and Jieyu Zhang and Huanxin Sheng and Taiwei Shi and Gupta Rahul and Yang Liu and Ranjay Krishna and Jian Kang and Jieyu Zhao},year={2026},eprint={2603.10178},archivePrefix={arXiv},primaryClass={cs.CV},url={https://arxiv.org/abs/2603.10178},}
Apache License 2.0
10 |
stp_patch_level | true |
use_raw_frames_in_stp | true |
use_ttp | true |
ttp_threshold | 0.9999 |
ttp_similarity_metric | cosine |