GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
prithivMLmods
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
prithivMLmods
Model Tree
prithivMLmods/VisionGuardrail-9B
Input Modalities
Output Modalities
Supported Functionality
Quantization was performed using llmcompressor with the following recipe:
yaml
default_stage:default_modifiers:QuantizationModifier:targets: [Linear]ignore: ['re:.*lm_head', 're:.*embed_tokens$', 're:.*visual.*', 're:.*model.visual.*','re:.*linear_attn.*']scheme: FP8_DYNAMICbypass_divisibility_checks: falserequires_calibration_data: false
Linear layers are quantized to FP8 using dynamic per-tensor activation scaling. The lm_head, embedding table, vision components (visual), and linear_attn layers are excluded from quantization and remain at full precision.
This approach reduces the overall model size and memory requirements while preserving the vision-language representation and content-safety classification capabilities of the base model.
| Base model | prithivMLmods/VisionGuardrail-9B |
| Underlying model | Qwen/Qwen3.5-9B |
| Quantization scheme | FP8_DYNAMIC (Linear layers only) |
| Format | compressed-tensors |
| Calibration required | No |
| Excluded from quantization | lm_head, embed_tokens, visual, linear_attn |
| Developer | prithivMLmods |
| Quantized by | prithivMLmods |
| Architecture | Multimodal decoder-only language model |
| Parameters | 9B |
| Base model | Qwen/Qwen3.5-9B |
| Training dataset | ImageShield-Guardrail-Pro |
| Task | Multimodal content-safety classification |
| Input | Image + text prompt |
VisionGuardrail-9B-FP8 is intended for multimodal content-safety research and visual content classification.
The model can be used to analyze images for potentially sensitive or unsafe visual content, including:
The model is designed to provide a Safe or Unsafe classification based on the visual content and accompanying prompt.
The FP8 checkpoint can be loaded using the Transformers ecosystem with support for compressed-tensors quantization.
python
from transformers import AutoProcessor, AutoModelForImageTextToTextfrom PIL import Imagemodel_id = "prithivMLmods/VisionGuardrail-9B-fp8"processor = AutoProcessor.from_pretrained(model_id)model = AutoModelForImageTextToText.from_pretrained(model_id,device_map="auto",)image = Image.open("image.jpg")messages = [{"role": "user","content": [{"type": "image"},{"type": "text","text": "Classify this image as Safe or Unsafe and explain the safety classification.",},],}]text = processor.apply_chat_template(messages,tokenize=False,add_generation_prompt=True,)inputs = processor(text=text,images=image,return_tensors="pt",).to(model.device)outputs = model.generate(**inputs,max_new_tokens=512,)response = processor.batch_decode(outputs,skip_special_tokens=True,)[0]print(response)
Note: Refer to the base VisionGuardrail-9B model card for the recommended prompt format and inference behavior.
A simple classification prompt can be used:
text
Analyze the provided image for content safety.Classify the image as either:- Safe- UnsafeProvide a concise explanation for the classification.
For production applications, it is recommended to define a strict output schema and validate the generated response before consuming the classification programmatically.
FP8 inference is recommended on GPUs with native FP8 support, such as NVIDIA Hopper and Blackwell architectures.
The quantized model can also be used on other supported GPUs where FP8 weights are dequantized during execution, although performance and memory characteristics may differ.
For best performance:
compressed-tensors support.bfloat16 for non-quantized components where supported.The complete quantization configuration is:
yaml
default_stage:default_modifiers:QuantizationModifier:targets: [Linear]ignore:- 're:.*lm_head'- 're:.*embed_tokens$'- 're:.*visual.*'- 're:.*model.visual.*'- 're:.*linear_attn.*'scheme: FP8_DYNAMICbypass_divisibility_checks: falserequires_calibration_data: false
Keeping the vision components and selected attention layers in higher precision is intended to preserve multimodal representation quality and classification behavior.
VisionGuardrail-9B-FP8 inherits the limitations of VisionGuardrail-9B.
The model is an experimental content-safety classifier and should not be treated as a definitive safety system. Classification errors, false positives, and false negatives may occur.
FP8 quantization may introduce small numerical differences compared with the original BF16 checkpoint. Downstream performance should therefore be evaluated against the original model for production deployments.
The model should be used as one component of a broader content-moderation pipeline rather than as the sole safety mechanism for high-risk applications.
This model is intended for research, evaluation, and responsible content-safety applications.
Users are responsible for ensuring that their deployment complies with applicable laws, regulations, platform policies, and organizational safety requirements.
For the full safety guidance, intended use, limitations, and responsible AI considerations, refer to the VisionGuardrail-9B model card.
VisionGuardrail-9B-FP8 follows the licensing terms of the underlying Qwen/Qwen3.5-9B model and the VisionGuardrail-9B release.
Please review the applicable licenses before using the model in production or redistribution.
| Output | Safe / Unsafe classification with reasoning |
| Quantization | FP8 Dynamic |
| Format | compressed-tensors |