GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
XinNUS
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
XinNUS
Model Tree
Qwen/Qwen3-VL-4B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This is the Qwen-initialized temporal model used for the Qwen comparison on TimeLens and the main DOVE captioning result. It is distinct from the OMTG-initialized temporal model and from the ECCV image-mask model.
| Benchmark | Metrics | Result (%) |
|---|---|---|
| Charades-STA | R1@0.3 / R1@0.5 / R1@0.7 / tIoU | 74.8 / 58.6 / 30.7 / 51.9 |
| DOVE | TEC / Temp-Cov / Clip-Complete | 33.3 / 33.4 / 14.6 |
Numbers follow the paper's evaluation protocols. Minimal inference examples are not substitutes for the official benchmark preprocessing and scoring.
Use Transformers with Qwen3-VL support and install qwen-vl-utils plus a
working video decoder. The weights are stored in bfloat16.
python
import torchfrom transformers import AutoModelForImageTextToText, AutoProcessorfrom qwen_vl_utils import process_vision_info model_id = "XinNUS/Generalized-CycleGRPO-Temporal-4B"model = AutoModelForImageTextToText.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto").eval()processor = AutoProcessor.from_pretrained(model_id) messages = [{"role": "user", "content": [ {"type": "video", "video": "example.mp4", "fps": 2.0}, {"type": "text", "text": ( "Please provide a detailed description of what happens in this video " "during 12.0 - 18.0 seconds. Describe only the specified occurrence." )},]}]text = processor.apply_chat_template( messages, tokenize=False, add_generation_prompt=True)images, videos, video_kwargs = process_vision_info( messages, return_video_kwargs=True)inputs = processor( text=[text], images=images, videos=videos, padding=True, return_tensors="pt", **video_kwargs).to(model.device)with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=256, do_sample=False, use_cache=True )print(processor.batch_decode( output[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])Replace the video and interval with your own input. Long videos may require a suitable frame/pixel budget to fit GPU memory; the complete source video must remain the input for DOVE-style interval-conditioned captioning.
Reconstructability does not guarantee factual completeness, and descriptions can omit details or confuse repeated events. Do not use generated descriptions as verified evidence without human review. This model inherits the Qwen3-VL architecture and tokenizer. Temporal data and task design draw on OMTG. See NOTICE and LICENSE for attribution and license terms.
Xin Zhang, Haochen Wang, Beibei Lin, Zhaoxiang Zhang, Robby T. Tan, Jason Li, and Dacheng Tao.
import torchfrom transformers import AutoModelForImageTextToText, AutoProcessorfrom qwen_vl_utils import process_vision_info model_id = "XinNUS/Generalized-CycleGRPO-Temporal-4B"model = AutoModelForImageTextToText.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto").eval()processor = AutoProcessor.from_pretrained(model_id) messages = [{"role": "user", "content": [ {"type": "video", "video": "example.mp4", "fps": 2.0}, {"type": "text", "text": ( "Please provide a detailed description of what happens in this video " "during 12.0 - 18.0 seconds. Describe only the specified occurrence." )},]}]text = processor.apply_chat_template( messages, tokenize=False, add_generation_prompt=True)images, videos, video_kwargs = process_vision_info( messages, return_video_kwargs=True)inputs = processor( text=[text], images=images, videos=videos, padding=True, return_tensors="pt", **video_kwargs).to(model.device)with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=256, do_sample=False, use_cache=True )print(processor.batch_decode( output[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0])