GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
simaai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
simaai
Model Tree
microsoft/Phi-4-mini-instruct
Input Modalities
Output Modalities
Supported Functionality
| Component | Method | Weight format | Details |
|---|---|---|---|
| Decoder Linear layers | AutoRound | symmetric INT4, group size 256 | 128 modules; W4A16 |
lm_head | GPTQ | symmetric INT8, per-channel | static activation ordering; block size 128; dampening 0.01 |
| Activations | — | FP16/BF16 runtime activation path | not activation-quantized by this checkpoint |
AutoRound calibration used 512 fixed-order samples from NeelNanda/pile-10k
(seed 42), with a sequence length of 1024 tokens. AutoRound used 200 iterations,
batch size 1, and device ID 0.
Perplexity is measured with lm-eval on EleutherAI/wikitext_document_level,
wikitext-2-raw-v1, using the complete split (62 original / 62 effective
documents) and the rolling word-perplexity metric. The comparison uses identical
evaluation settings for both models.
| Checkpoint | Word perplexity | Status |
|---|---|---|
microsoft/Phi-4-mini-instruct | 11.7135 | Full run: 2026-07-15 |
| This Autoround Safetensors checkpoint | 12.6569 | Full run: 2026-07-15 |
| Absolute degradation | +0.9434 | quantized - source; lower is better |
| Relative degradation | +8.054% | 100 * (quantized / source - 1) |
The raw full-run results are retained in
perplexity_results/full_wikitext/ in the quantization workspace.
The exact quantization script and effective recipe are included in this directory:
quantize.pyrecipe.yamlOn the quantization host, reproduce with:
bash
python quantize.py \--model-path /path/to/Phi-4-mini-instruct \--output-dir /path/to/new-output-directory
The quantization job completed successfully on an NVIDIA A100 80 GB GPU on 2026-07-15. Saved quantization scales were checked for NaN/Inf values.
Exact package and runtime versions are recorded in versions.txt. The source
model revision was not recorded in the local cache; pin an immutable Hugging
Face revision before publishing this checkpoint.
This repository contains the pre-LLiMa quantized Hugging Face artifact. Compile it for the target Sima.ai platform with the appropriate LLiMa release, for example:
bash
llima-compile /path/to/Phi-4-mini-instruct-Autoround-Safetensors
Treat any compiler output as a separate deployment artifact unless its release format and redistribution terms have been explicitly approved.
Quantization can affect generation quality, especially on tasks unlike the calibration data. Validate the checkpoint for your language, domain, prompt format, context length, and safety requirements before deployment.