GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
WiktorMatuszek
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
WiktorMatuszek
Model Tree
abacusai/Smaug-Mini
Input Modalities
Output Modalities
Supported Functionality
0.47.0compressed-tensors / mxfp4-pack-quantized--kv-cache-dtype fp8); the repacked checkpoint does not encode a KV-cache quantization scheme.The included repack_report.json records conversion checks over representative layers.
Hugging Face's automated safetensors metadata may display an 8-bit tag and a lower parameter total for this checkpoint because packed FP4 tensors are stored in uint8 containers. The model architecture remains the 27B Smaug-Mini architecture.
Example vLLM invocation:
bash
vllm serve WiktorMatuszek/smaug-mini-mxfp4 \ --kv-cache-dtype fp8 \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_codervLLM supports the mxfp4-pack-quantized compressed-tensors format. On supported SM100+ hardware it can use a true W4A4 path; otherwise the format can fall back to W4A16 through Marlin.
The full capability evaluation for the unquantized model is published on the Smaug-Mini model card. This repository does not claim an independent rerun of that benchmark suite. Quantization can change outputs, so evaluate the checkpoint on your own workload before deployment.
Apache-2.0, following the source checkpoint. Smaug-Mini is published by Abacus.AI; this quantization is an independent community conversion.