GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
local-inference-lab
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
local-inference-lab
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
| Component | Representation |
|---|---|
| MLP gate, up and down projections | NVFP4, 16-element blocks |
| Attention and gated-delta-network projections | MXFP8, 32-element blocks |
| LM head | MXFP8, 32-element blocks |
| Trained text normalization weights | FP32 masters |
| Token embeddings, GDN convolutions and dynamics | Original BF16 |
| Vision encoder and remaining source tensors | Unchanged |
Attention/GDN projections and the LM head were frozen in their MXFP8 representations during distillation. Packed NVFP4 and MXFP8 weights reconstruct to the same BF16 weight values used by the student during training. The tokenizer, chat template, generation configuration and multimodal processors are retained from the base model.
MLP activation scales use the p99.999 token-row maximum from 390,497,191 raw-text and chat tokens. Each token contributes its maximum absolute input value across channels; an exact BF16 histogram records these row maxima. Raw and chat histograms are pooled before selecting the quantile.
Within each of the 64 dense layers, gate (w1) and up (w3) have separate
scale tensors containing the same value. Down (w2) has an independent
scale. These are dense projections; there is no expert axis.
Calibration uses the trained weight representations with BF16 activations. The deployment configuration specifies calibrated NVFP4 MLP activations and dynamic MXFP8 attention/head activations. Serving kernels therefore introduce activation quantization beyond the training forward pass.
Hugging Face safetensors with ModelOpt mixed-precision quantization metadata. The runtime must support the base model architecture, NVFP4 dense linears and MXFP8 linears. Packed-weight reconstruction and export integrity are checked; downstream evaluation and serving-quality qualification are separate.
Apache 2.0, following the base model.