🚨 IMPORTANT — install path (must follow)
These weights are abliterated, but wrong install steps still produce stock refusals.
Required: use the GitHub one-shot script (do not improvise):
git clone https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
cd keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
bash scripts/oneshot_install_weights.sh # download (if needed) + verify + hub symlink
# multi-node: WORKERS="..." bash scripts/oneshot_install_weights.sh --fanout
Table with columns: Must, Must not| Must | Must not |
|---|
Run oneshot_install_weights.sh | Serve QuantTrio stock as if it were this repo |
verify_ablit_weights.py → PASS | Skip verify |
Hub name glm52-int4-int8mix → this tree | Random hub paths |
| Probe with thinking off for bypass checks | Claim “no ablit” from thinking-mode refusals |
Full rules: docs/IMPORTANT.md · docs/ONE_SHOT.md
GLM-5.2-Int4-Int8Mix-Abliterated
Late-only residual abliteration of the QuantTrio GLM-5.2 Int4–Int8 Mix MoE checkpoint, tuned for agent/chat use on multi-node DGX Spark (GB10) while keeping strong single-stream decode.
Responsible Use
This model has had safety refusals removed. That makes it useful for red-teaming, security research, evaluation, and unfiltered assistant tasks — and also removes guardrails a user must therefore supply themselves.
Access request (gated)
When requesting access on Hugging Face, provide:
Table with columns: Field, Purpose| Field | Purpose |
|---|
| Username | Your name or handle (form may default to your HF username) |
| Email | Contact email (form may default to your HF account email) |
| Reason for intended use | Brief description of planned use (e.g. red-teaming, evaluation, research, local assistant) |
Access is granted only after you complete these fields and accept the prohibited-use / responsibility terms.
Prohibited uses (access is gated on agreeing to these)
By requesting access, downloading, or using these weights, you agree:
- Anything involving the sexual exploitation or endangerment of minors.
- You must be of age 18 years or older to use and download this model.
- You agree any information generated that can cause harm in terms of generating recipe, knowledge to make any materials/substances is your own input and responsibility. You will be accountable for any harm/damage caused by your action/input.
- Content promoting self-harm or suicide.
- Generation of material that is illegal in your jurisdiction, or that targets real individuals for harassment, doxxing, or fraud.
- Any use prohibited by the upstream GLM license.
You are responsible for adding appropriate safety filtering, human review, and access controls for your deployment. The weights are provided as-is, with no warranty. The license is inherited from the upstream GLM base model — review and comply with it before use or redistribution.
By checking the gated-access agreement, you confirm you have read and accept these terms.
Credit — stock weights
This release is a derivative of QuantTrio’s Int4–Int8Mix pack. All credit for the mixed W4A16/W8A16 quantization, shard layout, and vLLM-oriented packaging goes to QuantTrio. We only applied a late-layer residual abliteration on top of that stock tree.
Please also cite / star the stock repo if you use this model:
https://huggingface.co/QuantTrio/GLM-5.2-Int4-Int8Mix
Table | |
|---|
| Ablit method | Prefill capture → SRA rank-1 (r=8) refusal direction → residual-write projection |
| Edits | self_attn.o_proj only, layers 65–77 (13 modules), λ=3.0 |
| Stock (untouched) | Layers 0–64, MTP / eh_proj, no late shared.down |
| Mean Δrel | ~0.040 |
Research / personal use. Base weights remain under their upstream licenses (see LICENSE and QuantTrio / zai-org terms). Abliteration does not remove all safety behavior; evaluate for your own risk tolerance and jurisdiction.
Measured standing (4× DGX Spark GB10)
Serve profile SPEED=1 (single stream), image vllm-node-tf5-glm52-b12x:probe-modded, 128k context, UTIL 0.86, MTP k=4, fp8_ds_mla KV, thinking off for probes:
Table with columns: Metric, Value| Metric | Value |
|---|
| True bypass (hard-refuse probe) | 32/32 |
| Garble | 0 |
| Harmless coherence | 6/6 |
| C1 structured (count/list) | ~29.7–30.2 tok/s |
| C1 essay / prose | ~17–18 tok/s (lower MTP accept; report separately) |
Full serve recipe: drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks (docs/STEP_BY_STEP_SPEED1.md).
Files
- Full Int4–Int8Mix checkpoint (124 shards + tokenizer + chat template), derived from QuantTrio/GLM-5.2-Int4-Int8Mix
ABLIT_META.json — edit recipe metadataREADME.base-quant.md — original quant README retained from the mix packaging
Only 13 late shards are physically re-quantized relative to stock; the rest match the QuantTrio mix tree (hardlink-friendly on disk).
vLLM notes (GB10 / multi-node)
This mix is prepared for vLLM + compressed-tensors. On GB10, use a TF5 / sm_121a image with sparse-MLA patches (see cookbook docs/IMAGE.md). Example standing launch:
MTP_K=4 SPEED=1 UTIL=0.86 MAX_MODEL_LEN=128000 \
./launch-keyspark.sh
# served-model-name glm-5.2 · port 8210 · nnodes=4 TP=4
Probe with chat_template_kwargs={"enable_thinking": false} when measuring C1 structured speed.
Quantization policy (from QuantTrio stock mix)
Unchanged from the QuantTrio Int4–Int8Mix release (see also README.base-quant.md):
Table with columns: Scope, Format| Scope | Format |
|---|
model.layers.0 | BF16 |
| Early linears | W8A16 g128 |
| MoE experts (mid/late) | W4A16 g128 |
| MTP block | W8A16 channelwise |
| Gates / norms / embeddings / indexer | FP32 / BF16 as stock |
Citation / lineage
- QuantTrio — GLM-5.2-Int4-Int8Mix stock quantized weights (primary base for this repo)
- zai-org/GLM-5.2 — upstream foundation model
- Abliteration recipe (this work) — late
o_proj L65–77 SRA residual write (drowzeys / keys)