GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
simaai
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
simaai
Model Tree
microsoft/Phi-4-mini-instruct
Input Modalities
Output Modalities
Supported Functionality
| Component | Method | Weight format | Details |
|---|---|---|---|
| Decoder Linear layers | AutoRound | symmetric INT4, G256 | lm_head excluded; 200 iterations; batch size 1 |
lm_head | GPTQ | symmetric INT8, per-channel | static act-order |
| Mixed-precision exceptions | — | source dtype | non-Linear parameters remain at source precision |
Calibration used HuggingFaceH4/ultrachat_200k, split train_sft[:512].
Conversations were rendered with the source chat template and packed into 512
sequences of 1,024 tokens without shuffling.
Full wikitext-2-raw-v1 evaluation used the wikitext lm-eval task with no
example limit and matched source/quantized settings on 2026-07-31.
| Checkpoint | Word perplexity | Status |
|---|---|---|
microsoft/Phi-4-mini-instruct | 11.713516 | Full run |
| This quantized checkpoint | 12.560528 | Full run |
| Absolute degradation | 0.847012 | Lower is better |
| Relative degradation | 7.231% | 100 * (quantized / source - 1) |
Finite-scale validation and a Transformers chat-generation smoke test passed.
This directory includes the model-specific quantize.py, recipe.yaml, and
versions.txt.
bash
python quantize.py \--model-path /path/to/models--microsoft--Phi-4-mini-instruct \--output-dir /path/to/Phi-4-mini-instruct-Autoround-Safetensors
Exact Python, CUDA, Torch, Transformers, llmcompressor, AutoRound, and
compressed-tensors versions are recorded in versions.txt.
This is the pre-LLiMa quantized Hugging Face artifact. Compile it separately for the target Sima.ai platform and keep compiler output separate.
Quantization quality can vary by language, domain, prompt format, context length, and deployment runtime. Validate the intended workload independently.