GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
simaai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
simaai
Model Tree
Qwen/Qwen2.5-VL-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Component | Method | Weight format | Targets |
|---|---|---|---|
| Decoder Linear layers | GPTQ | symmetric INT4, G256 | 196 |
lm_head | GPTQ | symmetric INT4, G256 | 1 |
| Vision encoder and merger Linear layers | GPTQ | symmetric INT8, per-channel | 162 |
| Mixed-precision Linear exceptions | — | None | 0 |
All GPTQ stages use static activation ordering, block size 128, and dampening fraction 0.01.
The decoder and output head use 512 deterministic, chat-templated samples from
HuggingFaceH4/ultrachat_200k train_sft at up to 1,024 tokens with seed 0.
The pinned dataset revision is
8049631c405ae6576f93f445c6b8166f76f5505a; the selected token-order SHA256 is
8fe3960c3b19e6963bd4e1c7b19b61aa24e9c619226627c90dc832c4f8424b3d.
The visual stack uses the first 512 deterministic examples from
lmms-lab/flickr30k test at up to 2,048 tokens with batch size 1. Exact
module targets are recorded in recipe.yaml.
The saved checkpoint contains all 359 intended quantized Linear targets, and all saved quantization scales were verified finite. It also passed Transformers load and generation smoke tests covering text, image-text, and multi-turn conversation inputs. LLiMa compilation and compiled-runtime testing were completed separately; compiler output is not included in this repository.
bash
python quantize.py --model-path /path/to/source-model --output-dir /path/to/new-output
The directory includes the exact quantize.py, effective recipe.yaml, and
environment versions.txt used for this artifact.
This is the pre-LLiMa quantized Hugging Face checkpoint. Keep generated LLiMa compiler output as a separate deployment artifact.
Quantization quality can vary by language, domain, prompt format, image type, and context length. Validate the intended deployment workload and safety requirements before use.