autoRiver

SAKI-InternVL3-2B-Instruct-bnb-8bit

README

License: apache-2.0

json
{
  "llm_int8_has_fp16_weight": false,
  "llm_int8_threshold": 6.0,
  "load_in_8bit": true
}

Load with trust_remote_code=True, device_map="auto", and torch_dtype="auto".

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Model Details

Model Provider

autoRiver

Model Tree

Base

OpenGVLab/InternVL3-2B-Instruct

Quantized

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated Endpoints