GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Weidows
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
Weidows
Model Tree
Saint-lsy/MedSAM-Agent-Qwen3-VL-8B-MedSAM2
Input Modalities
Output Modalities
Supported Functionality
MedSAM-Agent reformulates interactive medical image segmentation into a multi-step, autonomous, decision-making process. It drives a segmentation tool (SAM-family) using MLLM-generated tool calls (add_bbox, add_point, stop_action). This checkpoint is the trained Qwen3-VL-8B grounding model without the separate MedSAM2 segmentation checkpoint (that lives in the code repo as MedSAM2_latest.pt).
python
import torchfrom transformers import AutoModel, AutoProcessor, BitsAndBytesConfig model_id = "Weidows/MedSAM-Agent-Qwen3-VL-8B-MedSAM2-4bit" quant_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)model = AutoModel.from_pretrained(model_id, quantization_config=quant_config, device_map="auto")processor = AutoProcessor.from_pretrained(model_id)If you use Unsloth, just pass
load_in_4bit=True/FastVisionModel.from_pretrained(model_id, load_in_4bit=True).
Follow the official inference flow from the MedSAM-Agent repo — pass this model folder as --model-path:
bash
cd inferpython run_single_inference.py \ --img-path infer/demo/BTCV-0-106_CT_abdomen.png \ --target-description "right kidney in abdomen CT" \ --model-path /path/to/this/model \ --seg-checkpoint /path/to/MedSAM2_latest.pt \ --seg-model medsampython
from unsloth import FastVisionModelmodel, tokenizer = FastVisionModel.from_pretrained( "Saint-lsy/MedSAM-Agent-Qwen3-VL-8B-MedSAM2", max_seq_length=8192, load_in_4bit=True, device_map="auto",)model.save_pretrained("MedSAM-Agent-Qwen3-VL-8B-MedSAM2-4bit")tokenizer.save_pretrained("MedSAM-Agent-Qwen3-VL-8B-MedSAM2-4bit")Please cite the original work:
bibtex
@misc{liu2026medsamagentempoweringinteractivemedical, title={MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning}, author={Shengyuan Liu and Liuxin Bao and Qi Yang and Wanting Geng and Boyun Zheng and Chenxin Li and Wenting Chen and Houwen Peng and Yixuan Yuan}, year={2026}, eprint={2602.03320}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2602.03320},}import torchfrom transformers import AutoModel, AutoProcessor, BitsAndBytesConfig model_id = "Weidows/MedSAM-Agent-Qwen3-VL-8B-MedSAM2-4bit" quant_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)model = AutoModel.from_pretrained(model_id, quantization_config=quant_config, device_map="auto")processor = AutoProcessor.from_pretrained(model_id)cd inferpython run_single_inference.py \ --img-path infer/demo/BTCV-0-106_CT_abdomen.png \ --target-description "right kidney in abdomen CT" \ --model-path /path/to/this/model \ --seg-checkpoint /path/to/MedSAM2_latest.pt \ --seg-model medsamfrom unsloth import FastVisionModelmodel, tokenizer = FastVisionModel.from_pretrained( "Saint-lsy/MedSAM-Agent-Qwen3-VL-8B-MedSAM2", max_seq_length=8192, load_in_4bit=True, device_map="auto",)model.save_pretrained("MedSAM-Agent-Qwen3-VL-8B-MedSAM2-4bit")tokenizer.save_pretrained("MedSAM-Agent-Qwen3-VL-8B-MedSAM2-4bit")@misc{liu2026medsamagentempoweringinteractivemedical, title={MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning}, author={Shengyuan Liu and Liuxin Bao and Qi Yang and Wanting Geng and Boyun Zheng and Chenxin Li and Wenting Chen and Houwen Peng and Yixuan Yuan}, year={2026}, eprint={2602.03320}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2602.03320},}