GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
simaai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
simaai
Model Tree
Qwen/Qwen3-0.6B
Input Modalities
Output Modalities
Supported Functionality
| Component | Method | Weight format | Details |
|---|---|---|---|
| Decoder Linear layers | AutoRound | symmetric INT4, group size 256 | 196 modules; lm_head excluded; W4A16 |
lm_head | GPTQ | symmetric INT4, group size 256 | static activation ordering; block size 128; dampening 0.01 |
| Activations | — | FP16/BF16 runtime activation path | not activation-quantized by this checkpoint |
Calibration used 512 fixed-order samples from NeelNanda/pile-10k, with a
sequence length of 1024 tokens. AutoRound used 200 iterations, batch size 1,
and device ID 0.
Perplexity is measured with lm-eval on
EleutherAI/wikitext_document_level, wikitext-2-raw-v1, with no example
limit and the rolling word-perplexity metric. Source and quantized runs use
identical evaluator settings.
| Checkpoint | Word perplexity | Status |
|---|---|---|
Qwen/Qwen3-0.6B | 29.3376 | Full run: 2026-07-15 |
| This Autoround Safetensors checkpoint | 33.6043 | Full run: 2026-07-15 |
| Absolute degradation | +4.2667 | quantized - source; lower is better |
| Relative degradation | +14.543% | 100 * (quantized / source - 1) |
The raw full-run results are retained in perplexity_results/full_wikitext/
in the quantization workspace.
This directory includes the exact quantize.py, recipe.yaml, and
versions.txt used for this release. Reproduce after activating a compatible
environment:
bash
python quantize.py \--model-path /path/to/Qwen3-0.6B \--output-dir /path/to/new-output-directory
The quantization job completed successfully on an NVIDIA A100 80 GB GPU on
2026-07-15. Saved quantization scales were checked for NaN/Inf values. A
Transformers compressed-checkpoint smoke test also passed on an NVIDIA A100 80
GB GPU: 197 compressed modules, including a compressed lm_head, loaded and
completed greedy generation.
Exact package and runtime versions are recorded in versions.txt.
This directory contains the pre-LLiMa quantized Hugging Face artifact. Compile it for the target Sima.ai platform with the appropriate LLiMa release, for example:
bash
llima-compile /path/to/Qwen3-0.6B-Autoround-Safetensors
Treat compiler output as a separate deployment artifact unless its release format and redistribution terms have been explicitly approved.
Quantization can affect quality, especially for tasks unlike the calibration data. Validate this checkpoint for your language, domain, prompt format, context length, safety requirements, and deployment runtime before use.