GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
array
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
array
Model Tree
Qwen/Qwen2.5-VL-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Mull-Tokens are latent tokens that can be pre-trained to hold intermediate information in either image or text modalities so as to think towards the correct answer. Across four challenging spatial reasoning benchmarks, Mull-Tokens achieve a +3% average improvement and up to +16% on reasoning-heavy splits compared to the strongest baseline.
| Model | Description |
|---|---|
| array/Qwen2.5-VL-Mull | Mull-Tokens with multimodal warm-up |
| array/Qwen2.5-VL-MullGRPO | Mull-Tokens + GRPO reinforcement learning |
python
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom qwen_vl_utils import process_vision_infoimport torch # Choose model: "array/Qwen2.5-VL-Mull" or "array/Qwen2.5-VL-MullGRPO"MODEL_ID = "array/Qwen2.5-VL-Mull"NUM_LATENTS = 20 # Load model and processormodel = Qwen2_5_VLForConditionalGeneration.from_pretrained( MODEL_ID, torch_dtype=torch.bfloat16, attn_implementation="flash_attention_2", device_map="auto",)processor = AutoProcessor.from_pretrained(MODEL_ID) # Prepare your questionimage_path = "path/to/your/image.jpg"question = "If you stand at the X marked point and turn left, will the table be to your left or right? Please choose between the following answer choices: A. left. B. right. "question_type = "multiple choice" QUESTION_TEMPLATE_LATENT = ( "{Question}\n" "Please think about this question deeply. " "It's encouraged to include self-reflection or verification in the reasoning process. " "Provide your final answer between the <answer> </answer> tags.")TYPE_TEMPLATE = { "multiple choice": " Please provide only the single option letter (e.g., A, B, C, D, etc.) within the <answer> </answer> tags.", "numerical": " Please provide the numerical value (e.g., 42 or 3.14) within the <answer> </answer> tags.", "OCR": " Please transcribe text from the image/video clearly and provide your text answer within the <answer> </answer> tags.", "free-form": " Please provide your text answer within the <answer> </answer> tags.", "regression": " Please provide the numerical value (e.g., 42 or 3.14) within the <answer> </answer> tags.",}prompt = QUESTION_TEMPLATE_LATENT.format(Question=question) + TYPE_TEMPLATE[question_type] # Build messages with latent thinking tokensmessages = [ { "role": "user", "content": [ {"type": "image", "image": image_path}, {"type": "text", "text": prompt}, ], }, # IMPORTANT: Mull-Tokens requires latent thinking tokens before answer generation { "role": "assistant", "content": [ { "type": "text", "text": "<think>" + "<|latent_pad|>" * NUM_LATENTS + "</think>\n", } ], },] # Process inputstext = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)text = text.replace("<|im_end|>\n", "") # Remove end token so model continues generating image_inputs, video_inputs = process_vision_info(messages)inputs = processor( text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt",).to(model.device) # Generate responsewith torch.no_grad(): output_ids = model.generate( **inputs, max_new_tokens=512, do_sample=False, ) # Decode output (skip input tokens)generated_ids = output_ids[:, inputs["input_ids"].shape[1]:]response = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]print(response)vLLM serves this checkpoint with its native Qwen2.5-VL implementation — no
custom model code and no --trust-remote-code. The latent tokens are ordinary
entries in an extended vocabulary (vocab_size 151669) with trained embeddings,
and the latent-specific code paths in the training model are gated on a
training-only flag, so they never run during generation.
bash
vllm serve array/Qwen2.5-VL-Mull --max-model-len 32768The chat template pre-fills the assistant turn with <think> + 20 x
<|latent_pad|> + </think>, so ordinary chat requests get the latent block
automatically:
python
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")client.chat.completions.create( model="array/Qwen2.5-VL-Mull", messages=[{"role": "user", "content": [ {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}, {"type": "text", "text": question}, ]}], temperature=0, max_tokens=512, # optional: vary the latent budget, or set 0 to disable it for an ablation extra_body={"chat_template_kwargs": {"num_latents": 20}},)Check that the latent path is active. If you serve this model with a stock
Qwen2.5-VL chat template, the prompt ends at <|im_start|>assistant\n and the
model receives zero latent tokens. It still answers, so the mistake is easy to
miss — but it reverts to writing its reasoning out as text. On 24 SAT items the
same checkpoint averaged 8.9 output tokens with the latent block and
101.2 without. Verify with
list(output.prompt_token_ids).count(151665) == 20.
Reproducing the paper's numbers. The evaluations run
max_pixels=12845056, while preprocessor_config.json defaults to 401408:
bash
vllm serve array/Qwen2.5-VL-Mull --max-model-len 32768 \ --mm-processor-kwargs '{"min_pixels": 3136, "max_pixels": 12845056}'A recipe, serving scripts and an
HF-vs-vLLM parity harness live in
serving/.
bibtex
@misc{ray2025mulltokensmodalityagnosticlatentthinking, title={Mull-Tokens: Modality-Agnostic Latent Thinking}, author={Arijit Ray and Ahmed Abdelkader and Chengzhi Mao and Bryan A. Plummer and Kate Saenko and Ranjay Krishna and Leonidas Guibas and Wen-Sheng Chu}, year={2025}, eprint={2512.10941}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2512.10941},}from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessorfrom qwen_vl_utils import process_vision_infoimport torch # Choose model: "array/Qwen2.5-VL-Mull" or "array/Qwen2.5-VL-MullGRPO"MODEL_ID = "array/Qwen2.5-VL-Mull"NUM_LATENTS = 20 # Load model and processormodel = Qwen2_5_VLForConditionalGeneration.from_pretrained( MODEL_ID, torch_dtype=torch.bfloat16, attn_implementation="flash_attention_2", device_map="auto",)processor = AutoProcessor.from_pretrained(MODEL_ID) # Prepare your questionimage_path = "path/to/your/image.jpg"question = "If you stand at the X marked point and turn left, will the table be to your left or right? Please choose between the following answer choices: A. left. B. right. "question_type = "multiple choice" QUESTION_TEMPLATE_LATENT = ( "{Question}\n" "Please think about this question deeply. " "It's encouraged to include self-reflection or verification in the reasoning process. " "Provide your final answer between the <answer> </answer> tags.")TYPE_TEMPLATE = { "multiple choice": " Please provide only the single option letter (e.g., A, B, C, D, etc.) within the <answer> </answer> tags.", "numerical": " Please provide the numerical value (e.g., 42 or 3.14) within the <answer> </answer> tags.", "OCR": " Please transcribe text from the image/video clearly and provide your text answer within the <answer> </answer> tags.", "free-form": " Please provide your text answer within the <answer> </answer> tags.", "regression": " Please provide the numerical value (e.g., 42 or 3.14) within the <answer> </answer> tags.",}prompt = QUESTION_TEMPLATE_LATENT.format(Question=question) + TYPE_TEMPLATE[question_type] # Build messages with latent thinking tokensmessages = [ { "role": "user", "content": [ {"type": "image", "image": image_path}, {"type": "text", "text": prompt}, ], }, # IMPORTANT: Mull-Tokens requires latent thinking tokens before answer generation { "role": "assistant", "content": [ { "type": "text", "text": "<think>" + "<|latent_pad|>" * NUM_LATENTS + "</think>\n", } ], },] # Process inputstext = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)text = text.replace("<|im_end|>\n", "") # Remove end token so model continues generating image_inputs, video_inputs = process_vision_info(messages)inputs = processor( text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt",).to(model.device) # Generate responsewith torch.no_grad(): output_ids = model.generate( **inputs, max_new_tokens=512, do_sample=False, ) # Decode output (skip input tokens)generated_ids = output_ids[:, inputs["input_ids"].shape[1]:]response = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]print(response)from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")client.chat.completions.create( model="array/Qwen2.5-VL-Mull", messages=[{"role": "user", "content": [ {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}}, {"type": "text", "text": question}, ]}], temperature=0, max_tokens=512, # optional: vary the latent budget, or set 0 to disable it for an ablation extra_body={"chat_template_kwargs": {"num_latents": 20}},)vllm serve array/Qwen2.5-VL-Mull --max-model-len 32768 \ --mm-processor-kwargs '{"min_pixels": 3136, "max_pixels": 12845056}'@misc{ray2025mulltokensmodalityagnosticlatentthinking, title={Mull-Tokens: Modality-Agnostic Latent Thinking}, author={Arijit Ray and Ahmed Abdelkader and Chengzhi Mao and Bryan A. Plummer and Kate Saenko and Ranjay Krishna and Leonidas Guibas and Wen-Sheng Chu}, year={2025}, eprint={2512.10941}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2512.10941},}