Model Details
Table with columns: Attribute, Value| Attribute | Value |
|---|
| Base Model | Qwen3.5-VL-397B-A17B |
| Architecture | Qwen3.5 MoE (Mixture of Experts) |
| Total Parameters | 397B |
| Active Parameters | ~17B (10 of 512 experts per token) |
| Text Backbone | 60 layers, hidden=4096 |
| Vision Encoder | 27 layers, hidden=1152, patch=16 |
| Training | SFT on robot manipulation data |
| Max Context | 262,144 tokens |
| License | Apache 2.0 |
Capabilities
- Robotic Video VQA: Visual question answering on robot manipulation videos (multi-view supported)
- Fine-Grained Manipulation Annotation: Step-by-step captioning across 10 dimensions:
Table with columns: Dimension, What it captures| Dimension | What it captures |
|---|
| Action Sequence | Step-by-step execution order |
| Active Actor | Which arm / end-effector to use |
| Target Object | Object disambiguation |
| Initial Configuration | Starting state of objects and robot |
| Final Configuration | End state after manipulation |
| Contact & Approach | Where and how contact is made |
| Trajectory & Orientation | Motion path and tool orientation |
| Body Motion | Full-body or joint-level movement |
Benchmark Results
Evaluated on RoboFine-Bench, a comprehensive benchmark for robotic video understanding.
VQA (VLM4Robotics Benchmark)
Evaluated on 500 robot manipulation samples across 10 datasets.
Table with columns: Model, Overall, Gnd., Act., State| Model | Overall | Gnd. | | | Act. | | | | State | | |
|---|
| | AA | TO | IC | AS | C&A | T&O | BM |
Caption (VLM4Robotics Benchmark)
Fine-grained step decomposition of robot manipulation videos.
Table with columns: Model, Easy, Hard| Model | Easy | | | | Hard | | | |
|---|
| Overall | Cons. | Cov. | A-Hal. | Overall | Cons. | Cov. | A-Hal. |
| Qwen3-VL-Plus | 75.4 | 75.2 |
Settings: fps=4, max_frames=512, temperature=0.0, top_p=0.95, thinking=able(only RoboFine-VLM don't use thinking)
Quick Start
[!Important]
RoboFine-VLM is built on Qwen3.5-VL-397B-A17B. Inference efficiency and throughput vary significantly across frameworks.
We recommend using the latest framework versions to ensure optimal performance and compatibility.
Using RoboFine-VLM via the Chat Completions API (OpenAI SDK)
After serving the model with a framework like vLLM or SGLang, you can use the following code:
from openai import OpenAI
import httpx
client = OpenAI(
api_key="EMPTY",
base_url="http://localhost:8000/v1",
http_client=httpx.Client(timeout=httpx.Timeout(600.0, connect=60.0)),
)
messages = [{"role": "user", "content": [
{
"type": "video",
"video": [
"https://your-bucket.oss.aliyuncs.com/frame_0000.jpg",
"https://your-bucket.oss.aliyuncs.com/frame_0001.jpg",
],
"max_frames": 512,
},
{"type": "text", "text": "Describe the robot manipulation in this video."},
]}]
response = client.chat.completions.create(
model="RoboFine-VLM-opensource",
messages=messages,
temperature=0.0,
top_p=0.95,
max_tokens=32768,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)
Hardware Requirements
Table with columns: Precision, Model Weights, Minimum GPUs, Total VRAM| Precision | Model Weights | Minimum GPUs | Total VRAM |
|---|
| BF16 | ~752GB | 8× H200-141GB | 1128GB |
| FP8 | ~376GB | 8× A100/H100-80GB | 640GB |
- 8× H200-141GB: BF16 full precision, no quantization needed
- 8× A100/H100-80GB: Requires FP8 quantization (
--quantization fp8)
Training Data
Fine-tuned on FineVLA-Data: 47,159 human-verified trajectories with fine-grained instructions, generated by FineVLA-Tool and validated through human inspection.
Citation
@article{hu2026finevla,
title={FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies},
author={Hu, Xintong and Huang, Xuhong and Zhang, Jinyu and Yao, Yutong and Sun, Yuchong and Wang, Qiuyue and Li, Mingsheng and Xie, Sicheng and Liu, Yitao and Chen, Junhao and others},
journal={arXiv preprint arXiv:2605.27284},
year={2026}
}
