kitaniai

GLM-5.3-UNCENSORED-MXFP4

README

License: glm-5.3

Source and provenance

Immediate source: dealignai/GLM-5.3-UNCENSORED-FP8.
Exact source revision: 12170112d07e50d7dbe8e5ca1691f4fd8c1179e1.
The source model card identifies JANGQ-AI/GLM-5.3-FP8 as its base and zai-org/GLM-5.3 as the upstream model.
This is FP8 → MXFP4 requantization. Source FP8 weights are dequantized using their original block scales before MXFP4 conversion. It does not recover precision already lost in the FP8 source.
No additional training was performed. The tokenizer and chat template are inherited from the source.
Conversion completed: 2026-10-06T14:45:24.988739+00:00.

Measured checkpoint statistics

Table with columns: Item, Value
Item	Value
Source safetensors size	755.632 GB / 703.737 GiB
MXFP4 safetensors size	423.753 GB / 394.651 GiB
Size reduction vs supplied FP8 source	43.92%
Compression ratio vs supplied FP8 source	1.783×
Source tensor elements, excluding FP8 scales	753,329,940,480
MXFP4 weight elements	737,308,311,552
MXFP4 weight matrices	58,596
Higher-precision tensor elements	16,021,628,928

Sizes above count safetensors files, including their headers. Tensor-element counts include non-scale buffers and are not a separately audited trainable-parameter count. Overall storage includes higher-precision tensors and quantization metadata. Exact values are in quantization_stats.json; per-shard sizes and SHA-256 hashes are in artifact_manifest.json.

Quantization details

Toolkit: AMD Quark 0.13, PyTorch 2.14.1+rocm7.2, ROCm 7.2.53211.
Method: file-to-file round-to-nearest quantization; no calibration dataset, GPTQ, AWQ, or fine-tuning.
Weight format: FP4 E2M1 values, two values per byte, with one E8M0 scale per group of 32 consecutive weights.
Activation policy for quantized layers: dynamic MXFP4, as specified in config.json.
Quark recipe: LLMTemplate.get("glm_moe_dsa").get_config(scheme="mxfp4"), with *eh_proj additionally excluded to preserve the MTP auxiliary projection in BF16.
Excluded source FP8 linear weights are recovered to BF16. Existing floating-point excluded tensors retain their source dtype.
Hardware available: 8× AMD Instinct MI355X; this file-to-file FP8 conversion used one GPU (AMD Radeon Graphics), following Quark's supported single-device FP8 recovery path.

Architecture

Table with columns: Item, Source configuration
Item	Source configuration
Architecture	`GlmMoeDsaForCausalLM`
Model type	`glm_moe_dsa`
Hidden layers	78
Hidden size	6144
Routed experts	256
Experts selected per token	8
Shared experts	1
Initial dense layers	3

The configured context limit is inherited metadata, not a measured serving capacity. Actual context and memory requirements depend on the runtime, KV cache, batch size, and hardware.

Validation

Passed complete shard/index consistency, file-length, tensor-coverage, dtype, packing-shape, and scale-shape checks.
Passed independent FP4 decoding and finite-value checks on 64 matrices / 4,980,736 weight elements, sampled across the checkpoint.
Sampled weight reconstruction relative RMSE vs recovered FP8 source: 0.111689; absolute RMSE: 0.00184817. Sampling uses the first 16 rows of evenly spaced matrix names; this is not a random or exhaustive quality estimate.
End-to-end smoke inference was tested on October 7, 2026 with the pinned runtime below. Perplexity, downstream benchmark accuracy, sustained tokens/second, and long-context quality remain unmeasured. Upstream benchmark scores are not results for this quantization.

See sample_validation.json for individual matrices and conversion.json for environment and timing details.

Usage

Use a runtime supporting both GLM-5.3 / glm_moe_dsa and AMD Quark MXFP4. The intended serving path is vLLM with ROCm on supported AMD Instinct hardware. A generic Transformers installation or a runtime supporting only GPT-OSS MXFP4 is not sufficient evidence of compatibility.

Download the checkpoint:

bash

hf download kitaniai/GLM-5.3-UNCENSORED-MXFP4 --local-dir ./GLM-5.3-UNCENSORED-MXFP4

Tested ROCm runtime

The working container is pinned by digest, because nightly changes:

text

vllm/vllm-openai-rocm@sha256:c36857e9b1710ac01090c4ca4ecfa6fcca118638394ad43c01b13661d9d960f7

vLLM 0.31.1rc1.dev23+g43b4aaea3.rocm723, AITER 0.1.24.post1, PyTorch 2.13.0+git733fca1, ROCm 7.2.53211.
Tested on 4× MI355X, tensor parallelism 4, original W4A4 policy, native AITER MXFP4 kernels, and FULL_DECODE_ONLY graph capture.
Use the stock vLLM entrypoint in the script; no external serve_full.py wrapper is required for this tested image.

bash

HIP_VISIBLE_DEVICES=3,4,5,6 bash ./GLM-5.3-UNCENSORED-MXFP4/serve_rocm.sh ./GLM-5.3-UNCENSORED-MXFP4

The script binds to 127.0.0.1:8003, configures 131,072 maximum context and 32 sequences, and preserves the tested parser and graph settings. The context limit is configured, not a completed long-context validation. GPU families other than MI355X have not been validated.

Inference checks and known limitations

Using unchanged checkpoint revision c59bf9db2579ea83ae4eae15efaf2f1e9bb0888c:

Initial six prompts: correct arithmetic (323), capital, sorting values, and weather tool call; coherent code/prose. The sorting response used unwanted Markdown fences.
Thirty additional requests (three sequential six-prompt rounds, then a twelve-request concurrent batch): all 20 automatically scored arithmetic, capital, JSON and tool-call checks passed; no length-truncated responses or detected 12-word repetition. Ten code/prose responses are retained for review, not counted as benchmark passes.
A separate 32-request concurrent batch returned all 16 correct arithmetic values and all 16 expected weather tool calls. Exact integer-only formatting passed 14/16: two answers included extra text. All 32 requests completed normally.
vLLM reported 100.04 GiB per rank for model loading and 0.26 GiB per rank for graph capture; these exclude the large KV-cache reservation and are not total VRAM requirements.

On vLLM 0.30.0, the original FP8 source passed the same initial checks, while this MXFP4 checkpoint repeated itself under both graph and eager execution. Switching to the pinned newer runtime resolved that observed failure. This comparison changed multiple runtime libraries; the exact upstream fix has not been bisected. Disabling graphs or changing to BF16 activations on 0.30.0 was not a sufficient fix in our tests.

These are smoke tests, not a claim of lossless quantization, perfect instruction following, or production validation. See runtime_validation.json and the full runtime_smoke_results.json. Re-run the supplied checks against the local service:

bash

python runtime_smoke.py my-runpython runtime_concurrent32.py

Runtime references: vLLM Quark support and AMD Quark file-to-file quantization.

Reproduce

Install ROCm-enabled PyTorch matching the recorded version, then install amd-quark==0.13, transformers==5.16.0, safetensors==0.8.0, and accelerate.

bash

hf download dealignai/GLM-5.3-UNCENSORED-FP8 --revision 12170112d07e50d7dbe8e5ca1691f4fd8c1179e1 --local-dir ./sourcepython quantize.py --source ./source --output ./outputpython validate.py --source ./source --output ./output --hashpython sample_check.py --source ./source --output ./output

License and attribution

The upstream model uses the GLM-5.3 License, preserved verbatim in this repository. The immediate source repository is tagged MIT, but the upstream model includes its own license conditions; this release is not labeled MIT-only. See LICENSE and LICENSE_PROVENANCE.json. Credit to Z.ai for GLM-5.3, JANGQ-AI for the FP8 base, dealignai for the supplied source checkpoint, and AMD for Quark. Kitani performed this MXFP4 conversion and packaging.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

kitaniai

Model Tree

Base

dealignai/GLM-5.3-UNCENSORED-FP8

Quantized

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: glm-5.3

Source and provenance

Immediate source: dealignai/GLM-5.3-UNCENSORED-FP8.
Exact source revision: 12170112d07e50d7dbe8e5ca1691f4fd8c1179e1.
The source model card identifies JANGQ-AI/GLM-5.3-FP8 as its base and zai-org/GLM-5.3 as the upstream model.
This is FP8 → MXFP4 requantization. Source FP8 weights are dequantized using their original block scales before MXFP4 conversion. It does not recover precision already lost in the FP8 source.
No additional training was performed. The tokenizer and chat template are inherited from the source.
Conversion completed: 2026-10-06T14:45:24.988739+00:00.

Measured checkpoint statistics

Table with columns: Item, Value
Item	Value
Source safetensors size	755.632 GB / 703.737 GiB
MXFP4 safetensors size	423.753 GB / 394.651 GiB
Size reduction vs supplied FP8 source	43.92%
Compression ratio vs supplied FP8 source	1.783×
Source tensor elements, excluding FP8 scales	753,329,940,480
MXFP4 weight elements	737,308,311,552
MXFP4 weight matrices	58,596
Higher-precision tensor elements	16,021,628,928

Quantization details

Toolkit: AMD Quark 0.13, PyTorch 2.14.1+rocm7.2, ROCm 7.2.53211.
Method: file-to-file round-to-nearest quantization; no calibration dataset, GPTQ, AWQ, or fine-tuning.
Weight format: FP4 E2M1 values, two values per byte, with one E8M0 scale per group of 32 consecutive weights.
Activation policy for quantized layers: dynamic MXFP4, as specified in config.json.
Quark recipe: LLMTemplate.get("glm_moe_dsa").get_config(scheme="mxfp4"), with *eh_proj additionally excluded to preserve the MTP auxiliary projection in BF16.
Excluded source FP8 linear weights are recovered to BF16. Existing floating-point excluded tensors retain their source dtype.
Hardware available: 8× AMD Instinct MI355X; this file-to-file FP8 conversion used one GPU (AMD Radeon Graphics), following Quark's supported single-device FP8 recovery path.

Architecture

Table with columns: Item, Source configuration
Item	Source configuration
Architecture	`GlmMoeDsaForCausalLM`
Model type	`glm_moe_dsa`
Hidden layers	78
Hidden size	6144
Routed experts	256
Experts selected per token	8
Shared experts	1
Initial dense layers	3

The configured context limit is inherited metadata, not a measured serving capacity. Actual context and memory requirements depend on the runtime, KV cache, batch size, and hardware.

Validation

Passed complete shard/index consistency, file-length, tensor-coverage, dtype, packing-shape, and scale-shape checks.
Passed independent FP4 decoding and finite-value checks on 64 matrices / 4,980,736 weight elements, sampled across the checkpoint.
Sampled weight reconstruction relative RMSE vs recovered FP8 source: 0.111689; absolute RMSE: 0.00184817. Sampling uses the first 16 rows of evenly spaced matrix names; this is not a random or exhaustive quality estimate.
End-to-end smoke inference was tested on October 7, 2026 with the pinned runtime below. Perplexity, downstream benchmark accuracy, sustained tokens/second, and long-context quality remain unmeasured. Upstream benchmark scores are not results for this quantization.

See sample_validation.json for individual matrices and conversion.json for environment and timing details.

Usage

Download the checkpoint:

bash

hf download kitaniai/GLM-5.3-UNCENSORED-MXFP4 --local-dir ./GLM-5.3-UNCENSORED-MXFP4

Tested ROCm runtime

The working container is pinned by digest, because nightly changes:

text

vllm/vllm-openai-rocm@sha256:c36857e9b1710ac01090c4ca4ecfa6fcca118638394ad43c01b13661d9d960f7

vLLM 0.31.1rc1.dev23+g43b4aaea3.rocm723, AITER 0.1.24.post1, PyTorch 2.13.0+git733fca1, ROCm 7.2.53211.
Tested on 4× MI355X, tensor parallelism 4, original W4A4 policy, native AITER MXFP4 kernels, and FULL_DECODE_ONLY graph capture.
Use the stock vLLM entrypoint in the script; no external serve_full.py wrapper is required for this tested image.

bash

HIP_VISIBLE_DEVICES=3,4,5,6 bash ./GLM-5.3-UNCENSORED-MXFP4/serve_rocm.sh ./GLM-5.3-UNCENSORED-MXFP4

Inference checks and known limitations

Using unchanged checkpoint revision c59bf9db2579ea83ae4eae15efaf2f1e9bb0888c:

Initial six prompts: correct arithmetic (323), capital, sorting values, and weather tool call; coherent code/prose. The sorting response used unwanted Markdown fences.
Thirty additional requests (three sequential six-prompt rounds, then a twelve-request concurrent batch): all 20 automatically scored arithmetic, capital, JSON and tool-call checks passed; no length-truncated responses or detected 12-word repetition. Ten code/prose responses are retained for review, not counted as benchmark passes.
A separate 32-request concurrent batch returned all 16 correct arithmetic values and all 16 expected weather tool calls. Exact integer-only formatting passed 14/16: two answers included extra text. All 32 requests completed normally.
vLLM reported 100.04 GiB per rank for model loading and 0.26 GiB per rank for graph capture; these exclude the large KV-cache reservation and are not total VRAM requirements.

bash

python runtime_smoke.py my-runpython runtime_concurrent32.py

Runtime references: vLLM Quark support and AMD Quark file-to-file quantization.

Reproduce

Install ROCm-enabled PyTorch matching the recorded version, then install amd-quark==0.13, transformers==5.16.0, safetensors==0.8.0, and accelerate.

bash

hf download dealignai/GLM-5.3-UNCENSORED-FP8 --revision 12170112d07e50d7dbe8e5ca1691f4fd8c1179e1 --local-dir ./sourcepython quantize.py --source ./source --output ./outputpython validate.py --source ./source --output ./output --hashpython sample_check.py --source ./source --output ./output