GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
densereward
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
densereward
Model Tree
Qwen/Qwen3-VL-4B-Instruct
Input Modalities
Output Modalities
Supported Functionality
0.374.The model was trained with a fixed system prompt and a strict output
format. Use the same system prompt at inference (system_prompt.txt in this
directory). It instructs the model to emit exactly a single float in
[0.000, 1.000] (three decimals).
The user turn should contain the scene image plus the task description, in the same format used during training (image + short task text).
Requires transformers>=4.57, qwen_vl_utils>=0.0.14, torch, accelerate.
Tested environment: Python 3.12, CUDA 12.8,
torch==2.8.0+cu128,torchvision==0.23.0+cu128,transformers==5.2.0,accelerate==1.13.0,qwen-vl-utils==0.0.14.bash
conda create -n densereward python=3.12 -yconda activate denserewardpip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128pip install "transformers==5.2.0" "accelerate==1.13.0" "qwen-vl-utils==0.0.14" pillow numpy
python
import reimport torchfrom transformers import AutoModelForImageTextToText, AutoProcessorfrom qwen_vl_utils import process_vision_infoMODEL_DIR = "densereward/densereward-1frame" # or a local path to this checkpoint# The exact system prompt used in training ships with the model:SYSTEM_PROMPT = open(f"{MODEL_DIR}/system_prompt.txt").read().strip()model = AutoModelForImageTextToText.from_pretrained(MODEL_DIR, torch_dtype=torch.bfloat16, device_map="auto")processor = AutoProcessor.from_pretrained(MODEL_DIR)messages = [{"role": "system", "content": SYSTEM_PROMPT},{"role": "user","content": [{"type": "image", "image": "file:///path/to/frame.png"},{"type": "text", "text": "Task: put the black bowl on the plate."},],},]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)image_inputs, video_inputs = process_vision_info(messages)inputs = processor(text=[text],images=image_inputs,videos=video_inputs,padding=True,return_tensors="pt",).to(model.device)with torch.no_grad():out = model.generate(**inputs, max_new_tokens=8, do_sample=False) # greedygen = out[:, inputs.input_ids.shape[1]:]raw = processor.batch_decode(gen, skip_special_tokens=True)[0].strip()reward = float(re.search(r"[01](?:\.\d+)?", raw).group())print(raw, "->", reward)
Notes:
max_new_tokens only needs to be a handful of tokens (a float is short).[0, 1] and handle non-float output.The release includes a minimal args.json ({model_type: qwen3_vl, swift_version})
so ms-swift's PtEngine / TransformersEngine can auto-detect the model type for
this local directory (its architecture otherwise matches several swift registry
entries). Load it as the base model with no adapter:
python
from swift import TransformersEngine, RequestConfig, InferRequestengine = TransformersEngine("densereward/densereward-1frame") # no adapters
Pass the system prompt above as a {"role": "system", ...} message and the task
as the user turn (<image>{task}), matching training.
Compatibility note: ms-swift 4.2.x targets
transformers>=4.57,<5. Under a much newer transformers (e.g. 5.x) the swift template/prompt composition can misbehave even though weights load fine — prefer thetransformerspath above, or a swift-matched transformers version, for swift-based inference.
Apache License 2.0 (see LICENSE). The base model
Qwen/Qwen3-VL-4B-Instruct
is also Apache-2.0. This fine-tune was produced at the University of North
Carolina at Chapel Hill.
bibtex
@article{fang2026densereward,title={DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation},author={Fang, Yu and Dong, Wanxi and Liu, Jiaqi and Yang, Yue and Huo, Mingxiao and Mu, Yao and Yao, Huaxiu and Li, Li Erran and Szafir, Daniel and Ding, Mingyu},journal={arXiv preprint arXiv:2607.13033},year={2026}}