GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
WiktorMatuszek
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
WiktorMatuszek
Model Tree
abacusai/Smaug-Mini
Input Modalities
Output Modalities
Supported Functionality
0.47.0general/ptq/nvfp4_default-kv_fp8_castThe checkpoint is stored in ModelOpt's Hugging Face format and includes hf_quant_config.json.
Hugging Face's automated safetensors metadata may display an 8-bit tag and a lower parameter total for this checkpoint because packed FP4 tensors are stored in uint8 containers. The model architecture remains the 27B Smaug-Mini architecture.
Example vLLM invocation:
bash
vllm serve WiktorMatuszek/smaug-mini-nvfp4 \ --quantization modelopt_fp4 \ --kv-cache-dtype fp8 \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_codervLLM selects an NVFP4 linear backend according to the GPU and available kernels.
The full capability evaluation for the unquantized model is published on the Smaug-Mini model card. This repository does not claim an independent rerun of that benchmark suite. Quantization can change outputs, so evaluate the checkpoint on your own workload before deployment.
Apache-2.0, following the source checkpoint. Smaug-Mini is published by Abacus.AI; this quantization is an independent community conversion.