GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
vrfai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
vrfai
Model Tree
google/gemma-4-12B-it
Input Modalities
Output Modalities
Supported Functionality
This model was quantized using NVIDIA ModelOpt with the following configurations:
| Property | Value |
|---|---|
| Base model | google/gemma-4-12B-it |
| Quant method | NVIDIA ModelOpt (FP8 E4M3 - num_bits: (4, 3)) |
| Weight scheme | Per-channel (axis: 0) |
| Input activation | Dynamic Per-token (type: dynamic) |
| Calibration dataset | CNN DailyMail (512 samples, max_seq_len 1024) |
| Calibration algorithm | max |
| Size | ~15 GB (vs ~23 GB BF16) |
The following modules are kept in full precision (BF16) to preserve accuracy:
lm_headmodel.embed_vision*model.embed_audio*self_attn layers (layers 0–47)The recipes and scripts used to quantize this model can be found in the following repository: