GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
simaai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
simaai
Model Tree
google/gemma-4-E2B-it
Input Modalities
Output Modalities
Supported Functionality
| Component | Method | Weight format | Effective targets |
|---|---|---|---|
Decoder Linear layers and lm_head | GPTQ | symmetric INT4, group size 256, static act-order | 277 |
| Vision encoder Linear layers | GPTQ | symmetric INT8, per-channel, static act-order | 113 |
| Audio tower | none | source BF16 | 134 explicit exceptions |
| Vision/audio embedding projectors | none | source BF16 | 2 explicit exceptions |
The lm_head follows the default GPTQ INT4/G256 policy; no INT8 escalation was made. Calibration used lmms-lab/flickr30k test[:512], deterministic dataset order, 512 image-text samples, sequence length 2048, batch size 1, no concatenation, and no padding to maximum length. Exact targets and exceptions are saved in recipe.yaml.
Full MMStar used all 1,500 examples, VLMEvalKit commit 7055d3010c38ccb5dcae1bc9535ca19c7fe5d79f, deterministic generation, a matched letter-only system prompt, maximum 16 new tokens, SDPA, and exact_matching without an API judge.
| Checkpoint | Overall accuracy |
|---|---|
| Source | 35.6667% |
| Quantized | 29.7333% |
| Absolute change | -5.9333 percentage points |
| Relative change | -16.6355% |
Validation: MMStar inference passed (1,500/1,500). Raw results are under vlmeval_results/mmstar_full/.
Full matched WikiText-2 word perplexity (EleutherAI/wikitext_document_level, wikitext-2-raw-v1, no example limit; 2026-07-18): source 3362.9125; this checkpoint 618.3263; raw-text delta -2744.5862 (-81.61%). Independent native-logit scoring confirmed that the source's untemplated raw-text likelihood is intrinsically anomalous, so this lower PPL is not a quality improvement and must not be used as a release gate. It conflicts with the full MMStar regression. Investigation and raw evidence: releases/vlm-full-wikitext-perplexity/GEMMA4_PPL_INVESTIGATION.md and perplexity_results/vlm_full_wikitext/.
This directory contains the exact quantize.py, recipe.yaml, and versions.txt used for this artifact:
bash
python quantize.py \--model-path /path/to/source-model \--output-dir /path/to/new-quantized-model
text
Python: 3.13.2torch: 2.11.0+cu128CUDA: 12.8transformers: 5.10.1llm-compressor: 0.12.0auto-round: 0.13.0compressed-tensors: 0.17.1
This is the pre-LLiMa quantized Hugging Face artifact. LLiMa compiler output must remain separate unless its format and redistribution are approved. No Hugging Face upload is authorized for this batch.
Quantization quality can vary by language, visual domain, prompt format, context length, and downstream runtime. MMStar does not replace application-specific validation. Audio weights and modality projectors remain in source precision.