GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
simaai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
simaai
Model Tree
Qwen/Qwen2.5-VL-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Component | Method | Weight format | Targets |
|---|---|---|---|
| Decoder Linear layers | AutoRound | symmetric INT4, G256 | 196 |
lm_head | GPTQ | symmetric INT4, G256, static act-order | 1 |
| Vision encoder and merger Linear layers | GPTQ | symmetric INT8, per-channel, static act-order | 162 |
| Projectors / exceptions | — | No AutoRound target | See recipe.yaml |
Stage 1 uses deterministic NeelNanda/pile-10k text calibration (dataset
revision not captured; 512 samples, sequence length 1024, AutoRound 200
iterations, batch size 1). Stage 2 uses deterministic lmms-lab/flickr30k
(dataset revision not captured), test[:512], for 512 image-text samples
(sequence length 2048, batch size 1) used by the head and vision GPTQ stages.
| Checkpoint | MMStar overall accuracy | Status |
|---|---|---|
| Source | 57.4000% | Matched full MMStar, 2026-07-16 |
| Existing GPTQ decoder baseline | 50.1333% | Matched full MMStar, 2026-07-16 |
| This hybrid checkpoint | 56.4667% | Matched full MMStar, 2026-07-17 |
The hybrid checkpoint is -0.9333 percentage points from source and +6.3333 points over the existing GPTQ decoder baseline. The matched evaluation used VLMEvalKit commit 7055d301, greedy decoding, SDPA attention, identical image-pixel bounds, and max_new_tokens=128 for all three checkpoints.
Validation: Transformers image-text smoke test passed.
Full matched WikiText-2 word perplexity (EleutherAI/wikitext_document_level, wikitext-2-raw-v1, no example limit; 2026-07-18): source 11.5587; this checkpoint 12.2139; absolute degradation +0.6552; relative degradation +5.67%. Raw JSON evidence: perplexity_results/vlm_full_wikitext/qwen25_vl_7b_{source,autoround_w4g256_headgptq_w4_visiongptq_w8}_v3.json.
bash
python quantize.py --model-path /path/to/source-model --output-dir /path/to/new-output
The directory contains quantize.py, recipe.yaml, and versions.txt.
Exact Python, CUDA, Torch, Transformers, llmcompressor, AutoRound, and
compressed-tensors versions are recorded in versions.txt.
This is the pre-LLiMa quantized Hugging Face artifact. Compile it separately for Sima.ai hardware and keep compiler output separate. No upload is authorized for this VLM batch.
Quantization quality varies by language, visual domain, prompt format, context length, and runtime. MMStar and WikiText do not replace validation on the intended deployment workload.