jkim96

gemma-4-26B-A4B-it-DASHQ-INT4-g128

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Get help setting up a custom Dedicated Endpoints.

Talk with our engineer to get a quote for reserved GPU instances with discounts.

README

License: apache-2.0

Install

bash

pip install git+https://github.com/JaeminK/dashq.git

Load

python

from dashq import load_quantized
model, tokenizer = load_quantized("jkim96/gemma-4-26B-A4B-it-DASHQ-INT4-g128", device_map="auto")

Quantization

Table
FieldValue
Base modelgoogle/gemma-4-26B-A4B-it
PrecisionINT4, group size 128
Scale / zero dtypefloat16
Calibrationwikitext2, 128 samples x 2048
Size15.8828 GB · original 51.6120 GB · 3.2x compression

Benchmarks

Full zero-shot / few-shot results for every DASH-Q checkpoint: github.com/JaeminK/dashq#benchmarks

Model provider

jkim96

Model tree

Base

google/gemma-4-26B-A4B-it

Quantized

this model

Modalities

Input

Text, Image

Output

Text

Pricing

Dedicated Endpoints

View details

Supported Functionality

Model APIs

Dedicated Endpoints

Container

More information

Explore FriendliAI today