ressl

GRM-2.6-Plus-NVFP4

Deploy Dedicated

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Model Details

Model Provider

ressl

Model Tree

Base

OrionLLM/GRM-2.6-Plus

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

GRM-2.6-Plus-NVFP4 API & Inference Endpoint | FriendliAI

README

License: apache-2.0

Quantization

Quant config: ModelOpt NVFP4_DEFAULT_CFG
Scheme: ModelOpt standard NVFP4 dynamic 4-bit quantization with the preset's built-in exclusions for lm_head, output layers, routing gates, and convolutional linear-attention components.
Tooling: nvidia-modelopt via mtq.quantize and export_hf_checkpoint.
Calibration: 512 samples from cnn_dailymail, sequence length 512, batch size 2.

Runtime

Use a recent vLLM build with ModelOpt quantization support on NVIDIA Blackwell:

bash

vllm serve rressl/GRM-2.6-Plus-NVFP4 \  --quantization modelopt \  --max-model-len 262144 \  --gpu-memory-utilization 0.90 \  --kv-cache-dtype fp8 \  --attention-backend flashinfer \  --enable-auto-tool-choice \  --tool-call-parser qwen3_coder \  --reasoning-parser qwen3 \  --trust-remote-code

NVFP4 requires Blackwell-class NVIDIA hardware for the fast path.