GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
simaai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
simaai
Model Tree
Qwen/Qwen3-4B
Input Modalities
Output Modalities
Supported Functionality
| Component | Method | Format | Details |
|---|---|---|---|
| Decoder Linear layers | AutoRound | symmetric INT4, G256 | 252 modules; lm_head excluded; W4A16 |
lm_head | GPTQ | symmetric INT4, G256 | static actorder; block size 128; dampening 0.01 |
| Activations | — | FP16/BF16 | not activation-quantized |
Calibration used 512 fixed-order samples from NeelNanda/pile-10k, sequence
length 1024, AutoRound 200 iterations, batch size 1, and device ID 0.
Full wikitext-2-raw-v1 was evaluated through lm-eval's wikitext task,
with no example limit and matched settings for both checkpoints.
| Checkpoint | Word perplexity | Status |
|---|---|---|
Qwen/Qwen3-4B | 18.4609 | Full run: 2026-07-15 |
| This checkpoint | 19.3515 | Full run: 2026-07-15 |
| Absolute degradation | +0.8905 | quantized - source |
| Relative degradation | +4.824% | 100 * (quantized / source - 1) |
Raw JSON evidence is retained in perplexity_results/full_wikitext/ in this
workspace.
Included files: quantize.py, recipe.yaml, and versions.txt.
bash
python quantize.py --model-path /path/to/Qwen3-4B --output-dir /path/to/new-output-directory
The quantization job completed on an NVIDIA A100 80 GB GPU on 2026-07-15 and
saved scales passed the NaN/Inf check. A compressed-checkpoint smoke test also
passed on an NVIDIA A100 80 GB GPU: 253 compressed modules, including a
compressed lm_head, loaded and completed greedy generation.
This is a pre-LLiMa quantized Hugging Face artifact. Compile separately for Sima.ai with the appropriate LLiMa release. Quantization quality can vary by language, domain, prompt format, and context length; validate your deployment.