GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
XinNUS
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
XinNUS
Model Tree
zhouyik/Qwen3-VL-4B-SAMTok
Input Modalities
Output Modalities
Supported Functionality
CycleGRPO-4B is a Qwen3-VL-4B that emits SAMTok mask tokens (<|mt_...|>). Plain
text generation works with 🤗 Transformers directly; turning the mask tokens into
segmentation masks needs the VQ-SAM2 decoder from the CycleGRPO repo
(projects.transformers.vq_sam2), so clone and install it first:
bash
pip install "transformers>=4.57"git clone https://github.com/devinxzhang/CycleGRPO.gitcd CycleGRPO # run from the repo root so `projects.transformers.vq_sam2` importspip install -e .python
import torchfrom transformers import Qwen3VLForConditionalGeneration, AutoProcessor model_id = "XinNUS/CycleGRPO-4B"model = Qwen3VLForConditionalGeneration.from_pretrained( model_id, dtype="auto", device_map="auto").eval()processor = AutoProcessor.from_pretrained(model_id) messages = [{ "role": "user", "content": [ {"type": "image", "image": "figs/totoro.jpg"}, {"type": "text", "text": "Describe the image with interleaved segmentation " "masks for the corresponding parts of the answer."}, ],}]inputs = processor.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",).to(model.device) out = model.generate(**inputs, max_new_tokens=512, do_sample=False)text = processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]print(text) # answer text interleaved with <|mt_start|><|mt_XXXX|><|mt_YYYY|><|mt_end|> mask tokensThe <|mt_...|> tokens are decoded to masks by the VQ-SAM2 mask tokenizer. Use the
reference implementation in the CycleGRPO repo rather than re-deriving it — see
evaluation/groundingsuite/qwen3vl_groundingsuite_infer.py (or
evaluation/dlc_bench/inference.py), which build the decoder and run the decode loop:
python
from projects.transformers.vq_sam2 import VQ_SAM2, VQ_SAM2Config, SAM2Config# Those scripts also contain the `DirectResize` preprocessor, the mt-token parsing# (extract_mt_token_ids / fix_mt_format), and the `VQ_SAM2.forward_with_codes(...)`# decode step (codebook size 256, depth 2). Reuse them directly.The decoder weights — mask_tokenizer_256x2.pth and sam2.1_hiera_large.pt — come
from the base model Qwen3-VL-4B-SAMTok.
Released under Apache-2.0. Derived from Qwen3-VL-4B-SAMTok; use is also subject to the base model's license and terms.
pip install "transformers>=4.57"git clone https://github.com/devinxzhang/CycleGRPO.gitcd CycleGRPO # run from the repo root so `projects.transformers.vq_sam2` importspip install -e .import torchfrom transformers import Qwen3VLForConditionalGeneration, AutoProcessor model_id = "XinNUS/CycleGRPO-4B"model = Qwen3VLForConditionalGeneration.from_pretrained( model_id, dtype="auto", device_map="auto").eval()processor = AutoProcessor.from_pretrained(model_id) messages = [{ "role": "user", "content": [ {"type": "image", "image": "figs/totoro.jpg"}, {"type": "text", "text": "Describe the image with interleaved segmentation " "masks for the corresponding parts of the answer."}, ],}]inputs = processor.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",).to(model.device) out = model.generate(**inputs, max_new_tokens=512, do_sample=False)text = processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]print(text) # answer text interleaved with <|mt_start|><|mt_XXXX|><|mt_YYYY|><|mt_end|> mask tokensfrom projects.transformers.vq_sam2 import VQ_SAM2, VQ_SAM2Config, SAM2Config# Those scripts also contain the `DirectResize` preprocessor, the mt-token parsing# (extract_mt_token_ids / fix_mt_format), and the `VQ_SAM2.forward_with_codes(...)`# decode step (codebook size 256, depth 2). Reuse them directly.