Serving with vLLM
Requires a vLLM build with compressed-tensors NVFP4 support (loads via MarlinNvFp4LinearKernel on Blackwell):
vllm serve Hadidiz9/GUI-Owl-1.5-32B-Instruct-NVFP4 \
--served-model-name gui-owl-32b \
--max-model-len 32768 \
--trust-remote-code \
--kv-cache-dtype fp8 \
--limit-mm-per-prompt '{"image":5}' \
--allowed-local-media-path /
Notes
- Context length: 32768 tokens.
- The model consumes screenshots (images) as input for GUI-agent tasks; the vision preprocessor
config is included.
- Quantization was performed on an NVIDIA GB10.
License
Inherits the Apache-2.0 license of the base model mPLUG/GUI-Owl-1.5-32B-Instruct. All credit for
the original model belongs to the mPLUG / GUI-Owl authors. This repository only redistributes a quantized
copy of their weights.