GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
0xdfi
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
0xdfi
Model Tree
QuantTrio/GLM-5.2-Int4-Int8Mix
Input Modalities
Output Modalities
Supported Functionality
The complete checkpoint is uploaded in this repository: 124 model shards, four MTP shards, the Safetensors index, runtime configuration, tokenizer, and chat templates. After accepting the access terms, users can download the repository and load or serve it directly with a compatible runtime; rebuilding from a separate base model is not required. An agent connects to an inference server that has loaded the checkpoint—not to the raw model files themselves.
The repository also includes a tested deterministic builder for reconstructing this exact checkpoint from the pinned QuantTrio/GLM-5.2-Int4-Int8Mix base, plus technical documentation of the direction-recovery method, transplant formula, quantization codec, and validation criteria. That documentation records how this checkpoint was produced and provides a technical reference for adapting the process to another compatible compressed-tensors W8 group-128 variant with matching architecture and tensor layout. This is not an automated cross-variant workflow: the repository does not include a generic direction-extraction script or a drop-in builder, and the included builder and manifests are pinned to the named base revision.
This repository therefore provides both distribution paths:
Huihui-ai produced an abliterated GGUF of GLM-5.2 by suppressing refusal-related
weight directions in the attention output projections. This kit transplants
those directions into QuantTrio's compressed-tensors W8 pack-quantized format,
which loads natively with the transformers library.
In Huihui's language, this is an uncensored version of GLM-5.2 created with abliteration. This repository adapts that intervention to the pinned QuantTrio checkpoint rather than redistributing Huihui's GGUF weights.
Specifically:
2f0aff2760627f87faf16f8adc81caca7d7b10f6) against the
Unsloth stock GGUF
(pinned revision abc55e72527792c6e77069c99b4cb7de16fa9f23) via top-2
randomised SVD on the blk.{13..77}.attn_output.weight tensors.1d3bcfe5ec549ecd000fd80b37f191183842e983) using fresh
group-128 requantization and validated against 65 numerical gates.The resulting model is not byte-identical to the Huihui GGUF. The edit magnitude uses a uniform scale factor of 1.75, which was the minimum tested value that passed all 65 frozen W8 numerical gates. Factor 1.0 reproduced the original Huihui magnitude but fell below the group-W8 reliable edit resolution; factor 1.6 still failed layer 23.
This model has reduced safety filtering. It can generate sensitive, controversial, inappropriate, harmful, unsafe, or illegal content. It is not suitable for deployment to minors, public-facing services, or high-security contexts without additional controls. Users are solely responsible for all outputs and for compliance with local laws and ethical standards. Research, testing, and controlled internal use with monitoring and manual review are recommended. There are no default safety guarantees. No warranty or liability is provided.
The builder applies 65 independent frozen W8 gates (one per edited layer). Gates require:
All 65 gates passed. The worst-performing layer was layer 46:
Summary: 65 layers edited, 130 tensor payloads modified (weight_packed and
weight_scale), 177,439 tensors byte-identical to the pristine QuantTrio base.
These results are specific to this build and the test corpus used. They are not a universal guarantee.
A 32-question refusal corpus was evaluated:
Accept the repository access terms on Hugging Face, then download the complete checkpoint:
bash
hf download 0xdfi/GLM-5.2-QuantTrio-Abliterated \--local-dir ./GLM-5.2-QuantTrio-Abliterated
The download contains 124 model shards, four MTP shards, model.safetensors.index.json, runtime configuration, tokenizer files, and chat templates. Reserve at least 430 GB of free disk space for the download and additional space required by your runtime.
Point a GLM-5.2 inference runtime that supports QuantTrio's compressed-tensors INT4/INT8Mix checkpoint format at the downloaded directory. After the inference server is healthy, point your agent at that server's OpenAI-compatible endpoint and use the served model name. Raw weights are not themselves an agent endpoint; the inference server is the layer between the model files and the agent.
Runtime flags, tensor/context parallelism, memory limits, and maximum context depend on the target hardware. Do not copy the four-DGX-Spark production profile onto unrelated hardware without validating memory and backend compatibility.
You need approximately 820 GiB free disk space on a single filesystem.
bash
pip install -r requirements.txt
bash
hf download QuantTrio/GLM-5.2-Int4-Int8Mix \--revision 1d3bcfe5ec549ecd000fd80b37f191183842e983 \--local-dir ./base
bash
python build.py \--base-dir ./base \--output-dir ./output \--chunk-rows 512
The builder will:
output/BUILD_RECEIPT.json.PASS on success.Alternatively, let the builder download the base automatically:
bash
python build.py --download-base --output-dir ./output --chunk-rows 512
bash
python -c "import jsonr = json.load(open('output/BUILD_RECEIPT.json'))assert r['passed'], rprint('PASS: edited', r['edited_tensors'], 'tensors')"
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel = AutoModelForCausalLM.from_pretrained("./output", trust_remote_code=True)tokenizer = AutoTokenizer.from_pretrained("./output", trust_remote_code=True)
The output directory is a complete model tree in compressed-tensors W8 format, identical to the QuantTrio base except for the 130 edited tensor payloads.
markdown
model-00001-of-00124.safetensors... # 124 complete model shardsmodel-00124-of-00124.safetensorsmtp-00001-of-00004.safetensors... # 4 complete MTP shardsmtp-00004-of-00004.safetensorsmodel.safetensors.index.jsonconfig.jsonconfiguration.jsongeneration_config.jsontokenizer.jsontokenizer_config.jsonchat_template.jinjachat_template_original.jinjabuild.py # Agent-facing wrapper (constructs manifest, runs builder)repair_builder_v4.py # Reviewed fail-closed CPU-only builder (do not modify)requirements.txtrecipe/refusal_dirs.safetensors # FP32 ablation direction vectors, layers 13-77huihui_mag.json # Per-layer relative magnitudesmanifest.json # Provenance and numerical contractdocs/TECHNICAL_REPRODUCTION.md # Full acquisition and validation workflowtests/test_repair_builder_v4.py # Frozen builder teststest_public_wrapper.py # Wrapper testsLICENSENOTICE.mdAGENTS.md
The edit formula for each layer L is:
markdown
W*_L = W_qt,L - lambda_final_L * d_L (d_L^T W_qt,L)
where:
d_L is the unit-normalised ablation direction recovered from the Huihui/Unsloth diffqt_mag1_L = ||d_L^T W_qt,L||_2 / ||W_qt,L||_F (projection of QuantTrio matrix onto direction)lambda_match_L = m_L / qt_mag1_L (scale to match Huihui relative magnitude)lambda_final_L = 1.75 * lambda_match_Lm_L is the per-layer relative magnitude from recipe/huihui_mag.jsonAfter editing, the float32 result is requantized using the compressed-tensors group-128 W8 codec: fresh amax-based BF16 scales are stored before quantization, then weights are divided by the restored BF16 scale, clamped to [-128, 127], rounded to nearest even, converted to q+128 unsigned bytes, and packed little-endian into int32 lanes.
The builder is CPU-only, uses no network during the build phase, creates no
hardlinks or symlinks, writes through fsynced temporary files with atomic
rename, and operates under an exclusive fcntl lock with crash-recoverable
checkpoint state. See docs/TECHNICAL_REPRODUCTION.md for the full
acquisition and validation procedure.