GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
sanskar003
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
sanskar003
Model Tree
Qwen/Qwen3.5-9B
Input Modalities
Output Modalities
Supported Functionality
Qwen3.5-9B is a multimodal vision-language model from the Qwen team (Alibaba Group). It accepts interleaved image + text (and video) input and generates text, with a "thinking" reasoning mode enabled by default.
| Base model | Qwen/Qwen3.5-9B |
| Parameters | ~9B |
| Modality | Image-Text-to-Text (vision-language) |
| Architecture | Hybrid Gated DeltaNet + sparse MoE, with a vision encoder |
| Context length | 262,144 tokens native (extensible with RoPE scaling) |
| Languages | 201 languages and dialects |
| License | Apache 2.0 |
For full details on capabilities, benchmarks, and prompt formatting, see the base model card.
| Method | NVFP4 (4-bit floating point) |
| Format | compressed-tensors |
| Quantized modules | Linear layers of the language model |
| Kept in higher precision | Vision tower, router gates, and lm_head |
| Recommended hardware | NVIDIA Blackwell (FP4 tensor cores, sm_120) |
The vision tower is intentionally left unquantized to preserve image understanding quality.
bash
docker run --rm --gpus all -p 8000:8000 \ -v "$PWD:/models" \ vllm/vllm-openai:latest \ --model /models/Qwen3.5-9B-NVFP4 \ --served-model-name Qwen3.5-9B-NVFP4 \ --quantization compressed-tensors \ --max-model-len 32768Or directly from the Hub:
bash
docker run --rm --gpus all -p 8000:8000 \ vllm/vllm-openai:latest \ --model sanskar003/Qwen3.5-9B-NVFP4 \ --quantization compressed-tensorsbash
curl -s http://localhost:8000/v1/chat/completions \ -H 'Content-Type: application/json' -d '{ "model": "Qwen3.5-9B-NVFP4", "messages": [{"role": "user", "content": [ {"type": "text", "text": "Describe this image."}, {"type": "image_url", "image_url": {"url": "http://images.cocodataset.org/train2017/000000231895.jpg"}} ]}] }'compressed-tensors checkpoint.Qwen3_5ForConditionalGeneration
architecture.Released under Apache 2.0, inherited from the base model Qwen/Qwen3.5-9B. Please review and comply with the base model's license terms.
Please cite the original Qwen work — see the base model card for citation details.
docker run --rm --gpus all -p 8000:8000 \ -v "$PWD:/models" \ vllm/vllm-openai:latest \ --model /models/Qwen3.5-9B-NVFP4 \ --served-model-name Qwen3.5-9B-NVFP4 \ --quantization compressed-tensors \ --max-model-len 32768docker run --rm --gpus all -p 8000:8000 \ vllm/vllm-openai:latest \ --model sanskar003/Qwen3.5-9B-NVFP4 \ --quantization compressed-tensorscurl -s http://localhost:8000/v1/chat/completions \ -H 'Content-Type: application/json' -d '{ "model": "Qwen3.5-9B-NVFP4", "messages": [{"role": "user", "content": [ {"type": "text", "text": "Describe this image."}, {"type": "image_url", "image_url": {"url": "http://images.cocodataset.org/train2017/000000231895.jpg"}} ]}] }'