GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Vishva007
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Vishva007
Model Tree
XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
Input Modalities
Output Modalities
Supported Functionality
iters=1000, sym=True, group_size=32)W4A16 (4-bit weights, 16-bit activations)batch_size=2, gradient_accumulate_steps=4)mtp, mtp.fc, and visual kept in unquantized bfloat16 for exact multimodal and speculative execution.Run high-throughput inference with speculative decoding (MTP) and tool-calling enabled:
bash
vllm serve Vishva007/MiMo-V2.6-Distill-Qwen-9B-W4A16-GPTQ \ --port 8000 \ --host 0.0.0.0 \ --max-model-len 32768 \ --kv-cache-dtype fp8 \ --gpu-memory-utilization 0.90 \ --enable-prefix-caching \ --enable-chunked-prefill \ --max-num-batched-tokens 12288 \ --max-num-seqs 128 \ --trust-remote-code \ --dtype bfloat16 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3 \ --default-chat-template-kwargs '{"enable_thinking": false}' \ --speculative-config '{"method":"mtp","num_speculative_tokens":1}'When using modern or nightly vLLM releases (> v0.32.0), these checkpoints can be queried directly as structured vision-decision engines:
python
import base64import jsonimport httpx API_URL = "http://localhost:8000/v1/systemone"MODEL_ID = "Vishva007/MiMo-V2.6-Distill-Qwen-9B-W4A16-GPTQ"IMAGE_PATH = "cats.jpg" def to_data_uri(path: str) -> str: with open(path, "rb") as f: b64 = base64.b64encode(f.read()).decode("utf-8") return f"data:image/jpeg;base64,{b64}" payload = { "model": MODEL_ID, "state": "Examine the image and check if it contains a cat.", "images": [to_data_uri(IMAGE_PATH)], "questions": { "is_cat": { "type": "choice", "instructions": "Determine if a cat is clearly visible in the image.", "criteria": { "yes": "A cat is present", "no": "No cat is visible", }, } },} response = httpx.post( API_URL, json=payload, headers={"Authorization": "Bearer EMPTY"}, timeout=60.0,) print(json.dumps(response.json(), indent=2, ensure_ascii=False))One-click launch environments pre-configured with PyTorch, CUDA, and dependencies for fine-tuning or quantization.
🎁 Need GPU compute? Sign up via RunPod and get 5–500 in free credits when you add your first $10.
| Template | CUDA Version | Docker Image | Template ID | Deploy |
|---|---|---|---|---|
| PyTorch 2.14 (CUDA 12.6) | 12.6 | vishva123/cuda-12.6-pytorch-2.14-runpod | d7lxsa4w9m |  |
| PyTorch 2.14 (CUDA 13.0) | 13.0 | vishva123/cuda-13.0-pytorch-2.14-runpod | yk0y6j6rpg | |
| PyTorch 2.14 (CUDA 13.2) | 13.2 | vishva123/cuda-13.2-pytorch-2.14-runpod | gsp4gwx0nw | |
| Template | CUDA Version | Docker Image | Template ID | Deploy |
|---|---|---|---|---|
| PyTorch 2.13 (CUDA 12.6) | 12.6 | vishva123/cuda-12.6-pytorch-2.13-runpod | gmlupxnxfk | |
| PyTorch 2.13 (CUDA 13.0) | 13.0 | vishva123/cuda-13.0-pytorch-2.13-runpod | y3j8xvk4f4 | |
| PyTorch 2.13 (CUDA 13.2) | 13.2 | vishva123/cuda-13.2-pytorch-2.13-runpod | vigpissn5w | |
| Template | CUDA Version | Docker Image | Template ID | Deploy |
|---|---|---|---|---|
| PyTorch 2.12 (CUDA 12.6) | 12.6 | vishva123/cuda-12.6-pytorch-2.12-runpod | ctmz86zmf0 | |
| PyTorch 2.12 (CUDA 13.0) | 13.0 | vishva123/cuda-13.0-pytorch-2.12-runpod | qjko5yiwzi | |
| PyTorch 2.12 (CUDA 13.2) | 13.2 | vishva123/cuda-13.2-pytorch-2.12-runpod | ifg6xmye0f | |
bibtex
@misc{mimo2026v26, title={MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement}, author={{Xiaomi MiMo Team}}, year={2026}, howpublished={\url{[https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B)}},}vllm serve Vishva007/MiMo-V2.6-Distill-Qwen-9B-W4A16-GPTQ \ --port 8000 \ --host 0.0.0.0 \ --max-model-len 32768 \ --kv-cache-dtype fp8 \ --gpu-memory-utilization 0.90 \ --enable-prefix-caching \ --enable-chunked-prefill \ --max-num-batched-tokens 12288 \ --max-num-seqs 128 \ --trust-remote-code \ --dtype bfloat16 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3 \ --default-chat-template-kwargs '{"enable_thinking": false}' \ --speculative-config '{"method":"mtp","num_speculative_tokens":1}'import base64import jsonimport httpx API_URL = "http://localhost:8000/v1/systemone"MODEL_ID = "Vishva007/MiMo-V2.6-Distill-Qwen-9B-W4A16-GPTQ"IMAGE_PATH = "cats.jpg" def to_data_uri(path: str) -> str: with open(path, "rb") as f: b64 = base64.b64encode(f.read()).decode("utf-8") return f"data:image/jpeg;base64,{b64}" payload = { "model": MODEL_ID, "state": "Examine the image and check if it contains a cat.", "images": [to_data_uri(IMAGE_PATH)], "questions": { "is_cat": { "type": "choice", "instructions": "Determine if a cat is clearly visible in the image.", "criteria": { "yes": "A cat is present", "no": "No cat is visible", }, } },} response = httpx.post( API_URL, json=payload, headers={"Authorization": "Bearer EMPTY"}, timeout=60.0,) print(json.dumps(response.json(), indent=2, ensure_ascii=False))@misc{mimo2026v26, title={MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement}, author={{Xiaomi MiMo Team}}, year={2026}, howpublished={\url{[https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B)}},}