GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
malaiwah
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
malaiwah
Model Tree
Input Modalities
Output Modalities
Supported Functionality
qsrt_atoms_v1 storage with complete per-file SHA-256 manifests
and a fail-closed QSRT_COMPLETE.json marker.The expert allocation selected by the frozen calibration evidence is recorded
in each qsrt-layer-*.json sidecar. Aggregate allocation counts are:
| Allocation code | Experts |
|---|---|
R13=0,R2=0 | 2,797 |
R13=0,R2=1 | 1 |
R13=1,R2=0 | 11 |
R13=1,R2=1 | 5 |
R13=2,R2=1 | 1 |
R13=2,R2=2 | 1 |
The apples-to-apples baseline is the complete BF16 tensor set. All three rows
below cover the same 5,040,368,896 logical parameters and count only
Safetensors files; effective bpw is stored bytes times eight divided by that
parameter count, so it includes container and quantization metadata.
| Tensor payload | Bytes | GiB | Effective bpw | Relative to BF16 |
|---|---|---|---|---|
| BF16 source | 10,081,800,232 | 9.3894 | 16.0017 | baseline |
| Prior SIQ mixed | 3,102,116,152 | 2.8891 | 4.9236 | 69.23% smaller |
| This QSRT model | 2,909,352,104 | 2.7095 | 4.6177 | 71.14% smaller |
The whole-model rates include 611,183,872 non-routed parameters retained in BF16. Isolating the 4,429,185,024 routed-expert weights gives:
| Routed-expert format | Stored bytes | Nominal path bpw | Effective stored bpw |
|---|---|---|---|
| BF16 | 8,858,370,048 | 16.0000 | 16.0000 |
| Prior SIQ mixed (1,856 K3 / 960 K4 experts) | 1,879,717,272 | 3.3409 | 3.3951 |
| QSRT P24/P33 atoms | 1,686,953,224 | 3.0000 | 3.0470 |
QSRT is therefore 10.25% smaller than SIQ on the routed-expert component and 6.21% smaller on the compared tensor files. Package-level totals are not used for the comparison because tokenizer, card, and optional evaluation files are not model weights. The previous card's 7,593,020,594-byte BF16 row was not the complete BF16 tensor set and has been removed.
W4A8 and W4A16 use the same stored weights, so their loader weight storage is identical; W4A8 changes the decode execution path, not the checkpoint size.
The Hugging Face model API with blobs=true reported the following immutable
snapshot on 2026-08-08. Repository bytes sums every sibling's reported size;
Safetensors bytes sums only *.safetensors. These are observed repository
payloads, not parameter-count estimates.
| Artifact | Revision | Repository bytes | Safetensors bytes |
|---|---|---|---|
| Fruit Instruct BF16 | 678954f6 | 10,102,017,674 | 10,081,800,232 |
| Fruit Instruct prior mixed SIQ | 48452ef3 | 3,122,333,594 | 3,102,116,152 |
The three full-model rows ground real download/storage scale only. They are not used for Fruit percentage claims because Fruit has 5.04B parameters while the production model has roughly 754B. The apples-to-apples Fruit tensor comparison above remains the codec-size result.
evaluation/fruit-uniform-rate-sweep.json re-encodes the same authenticated expert sample at uniform
K2, K3, and K4, with fresh per-endpoint FP16 scales. It measured
16 of 19 predeclared assignments; 3 lacked
the minimum routed calibration support and were skipped rather than imputed.
These are pre-layer-deduplication expert-local endpoints, not package sizes.
| Endpoint | Mean bpw | Mean bytes/expert | Weight NMSE | Captured-H NMSE | Validation routed NMSE |
|---|---|---|---|---|---|
| K2 | 2.0469 | 402,432 | 0.082239 | 0.023648 | 0.055370 |
| K3 | 3.0469 | 599,040 | 0.020829 | 0.005855 | 0.013015 |
| K4 | 4.0469 |
Relative to K3, K2 is 3.948x / 4.039x / 4.254x on weight, captured-H, and validation-routed NMSE. K4 is 0.257x / 0.255x / 0.251x on the same metrics.
The completion seal covers every top-level package file and every regular file
under evaluation/. The sealed adjacent-rate report measures local routed
expert reconstruction on authenticated, document-disjoint calibration rows.
It does not establish chat quality, broad downstream task quality, or general
serving throughput.
The runtime is pinned to the reviewed commits below:
local-inference-lab/kquant#4,
encoded with KQuant revision d3355b1b1088c5c9fa2afbcc608cc627a8a42f69.local-inference-lab/b12x#129,
tested revision 84df431be79d6897fb3872516a7da39139e4905b.local-inference-lab/vllm#269,
tested revision 056c3e1f46c1870378d266f4ee5fa9049309bc04.The derived image starts from the content-addressed public base
docker.io/voipmonitor/vllm@sha256:3230c25ff95f8678a8eeb52a463f0d3b9f96f6ad550418cc51ea12177a55b41c
hard-coded by Dockerfile.fruit-qsrt. It installs the exact B12X checkout,
copies the base's compiled vLLM extensions into the reviewed source tree, and
seals the exact runtime package bytes. MODEL_REVISION resolves the Hub branch
once; hf download then uses the resulting immutable commit SHA.
bash
git clone https://github.com/malaiwah/vllm-voipmonitor.git vllm-fruitgit -C vllm-fruit checkout --detach 056c3e1f46c1870378d266f4ee5fa9049309bc04docker build \--file vllm-fruit/Dockerfile.fruit-qsrt \--build-arg VLLM_REVISION=056c3e1f46c1870378d266f4ee5fa9049309bc04 \--build-arg B12X_REVISION=84df431be79d6897fb3872516a7da39139e4905b \--tag fruit-qsrt:056c3e1f46c1870378d266f4ee5fa9049309bc04 \vllm-fruitMODEL_REVISION="$(curl -fsSL https://huggingface.co/api/models/malaiwah/GLM-5.2-QSRT-Fruit-Instruct-exact \| python3 -c 'import json,sys; print(json.load(sys.stdin)["sha"])')"MODEL_DIR="Fruit-QSRT-${MODEL_REVISION}"test ! -e "${MODEL_DIR}"hf download malaiwah/GLM-5.2-QSRT-Fruit-Instruct-exact \--revision "${MODEL_REVISION}" \--local-dir "${MODEL_DIR}"docker run --rm --gpus '"device=0"' --shm-size=16g \--read-only \--tmpfs /tmp:rw,exec,nosuid,size=8g \--tmpfs /cache:rw,exec,nosuid,size=16g \--tmpfs /root/.cache:rw,nosuid,size=1g \--publish 8000:8000 \--volume "$PWD/${MODEL_DIR}:/model:ro" \--env MODEL=/model \fruit-qsrt:056c3e1f46c1870378d266f4ee5fa9049309bc04
The qualified path is SM120 with CUDA 13.2.1 and PyTorch 2.12.0+cu132 in the
content-addressed base, plus nvidia-cutlass-dsl == 4.6.0 in the derived image.
The launcher rejects extra vLLM arguments and any value other than TP1,
max_num_seqs=1, max_model_len=4096, and
max_num_batched_tokens=4096 before importing the GPU runtime. The current
B12X sparse-prefill backend requires single-request prefill chunks.
The runtime manifest is an integrity check rooted in the trusted immutable
image, not an independent signature. The build host and container operator
remain trusted. Run the container read-only, keep writable tmpfs mounts outside
/opt/vllm-fruit, /opt/b12x-fruit, and /opt/fruit-runtime, and mount the
authenticated model read-only as shown above.
W4A16 is used for prefill and any row count above the W4A8 decode ceiling. W4A8 is selected for decode-sized batches of at most 16 rows. Unsupported shapes, activation modes, metadata, or incomplete manifests fail closed.
malaiwah/GLM-5.2-SIQ-Fruit-Instruct-bf16
at immutable revision 678954f65e056a0f508e21eeb9251c655bb9463f.safetensors_manifest) SHA-256:
8f23aed5e9b12000ed103a76da772a20730ca53ab7e352d6cb94da2709165245.24b290abaddb9eff8d7328a2a22f3c33bb7f6f45b77692393b3549ba0fded0a2.be944c8dfc5b550319d26bc2899f0d2ea3f4ca81275ba53034f0cc8ef7b4e9c5.MANIFEST.sha256, , ,
, and bind the published
package to the source and encoder fingerprints.84 for 84 * 3 / 2 instead of 126, and produced incoherent Chinese text; both honored a short named-system instruction. This is protocol evidence, not a representative benchmark, and no downstream task-quality claim is made.max_num_seqs=1 protocol is not a general throughput benchmark.max_num_seqs=1.The packaged model files are MIT, matching the authenticated Fruit BF16 source license. B12X and vLLM are Apache-2.0. KQuant is not redistributed in this model repository and remains subject to its upstream repository licensing.
b4734de4 |
| 1,506,693,036,946 |
| 1,506,667,387,408 |
| Full GLM-5.2 FP8 | ba978f7d | 755,663,676,164 | 755,632,050,320 |
| Full GLM-5.2 NVFP4 | aec724e8 | 464,874,323,992 | 464,823,042,096 |
| 795,648 |
| 0.005351 |
| 0.001493 |
| 0.003263 |
qsrt-manifest.json.qsrt-source-evidence.jsonqsrt-calibration-evidence.jsonQSRT_COMPLETE.json