GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
HawkBearPig
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
HawkBearPig
Model Tree
zai-org/GLM-5.3-BF16
Input Modalities
Output Modalities
Supported Functionality
glm_moe_dsa folds the smoothing scale of
post_attention_layernorm into the norm but does not rescale the MoE router (mlp.gate), and likewise
leaves the DSA indexer's wk / weights_proj / wq_b uncompensated in the 18 layers that own an indexer.
As saved by llm-compressor those modules receive x / s, and the per-layer error is 5–10× worse than plain
round-to-nearest. Every MoE shard here has the affected weight columns rescaled by the recovered scale
(original_norm / smoothed_norm), which restores the layer to slightly better than round-to-nearest. Corrected
shards carry the safetensors metadata key fold_corrected. The defect and fix are described in the issue draft
shipped with this card's tooling.Relative RMS error against the bf16 layer on 32 × 2048 real, held-out tokens. "Contribution" is the error of what the layer adds (output minus input); "residual" is the error of the full residual stream leaving the layer, which is the quantity that propagates.
| layer | variant | contribution | residual |
|---|---|---|---|
| 3 | this checkpoint (AWQ int4 g64, corrected) | 2.38 % | 0.53 % |
| 3 | RTN int4 g64 (no AWQ) | 2.40 % | 0.54 % |
| 3 | AWQ as llm-compressor saved it (uncorrected, g128) | 12.5 % | 2.80 % |
| 3 | fp8 e4m3 g128 on every projection, for reference | 2.46 % | 0.55 % |
The contribution metric grows with depth because each layer's addition shrinks relative to the residual stream; the residual error rises from 0.5 % (layer 3) to 2.3 % (layer 63). On this model the int4/int8 mix lands at the error of an all-fp8 recipe at half the bytes. AWQ's advantage over round-to-nearest is 0.9 / 4.8 / 1.2 / 0.4 / 0.02 % of the error at layers 3 / 6 / 21 / 42 / 63: real in the shallow indexer layer, negligible at depth. Per layer, this checkpoint and the round-to-nearest companion are equivalent.
No end-to-end evaluation yet (perplexity, benchmarks): the numbers above are per-layer. Treat this as a carefully validated but not yet benchmarked checkpoint.
A companion checkpoint with the same recipe but plain round-to-nearest instead of AWQ (no smoothing, no fold
correction needed) is at HawkBearPig/GLM-5.3-Int4-Int8Mix-RTN-g64.
Standard compressed-tensors layout: model.safetensors.index.json, per-layer shards layer-NNN.safetensors,
passthrough.safetensors (embeddings, norms, lm_head, MTP block), config.json with quantization_config,
tokenizer files. Any loader that understands glm_moe_dsa and compressed-tensors pack-quantized W4A16/W8A16
should read it; only DGPP has been exercised. vLLM / transformers loading is untested.
Tooling: llm-compressor 0.13.0, compressed-tensors 0.18.0, transformers 5.14.1, torch 2.13.0 (CUDA 13), one B200.
License: MIT, inherited from GLM-5.3.
| 6 (owns an indexer) | this checkpoint | 3.27 % | 1.50 % |
| 6 | RTN int4 g64 | 3.44 % | 1.58 % |
| 6 | AWQ uncorrected | 30.5 % | 14.0 % |
| 21 | this checkpoint | 10.95 % | 0.58 % |
| 21 | RTN int4 g64 | 11.09 % | 0.59 % |
| 42 (owns an indexer) | this checkpoint | 9.85 % | 2.00 % |
| 42 | RTN int4 g64 | 9.89 % | 2.01 % |
| 63 | this checkpoint | 12.50 % | 2.34 % |
| 63 | RTN int4 g64 | 12.50 % | 2.34 % |