GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
ala-la
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ala-la
Model Tree
wangzhang/Qwen3.6-35B-A3B-abliterated-v2
Input Modalities
Output Modalities
Supported Functionality
Compressed Qwen3.6-35B-A3B-abliterated-v2 (uncensored MoE, 70.2 GB FP16) to 19.94 GB using a 4-stage pipeline: AWQ + QuaRot + mixed 3/4-bit quantization + bias compensation.
PPL: +0.03% (28.377 → 28.387) measured in-memory on 50 calibration texts (~682 tokens).
| Metric | Value |
|---|---|
| FP16 PPL | 28.377 |
| Compressed PPL | 28.387 |
| Delta | +0.03% |
| FP16 size | 70.2 GB |
| Compressed size | 19.94 GB |
| Compression ratio | 3.5x |
This model uses custom QuantizedLinear layers not recognized by transformers.AutoModelForCausalLM. Loading via standard HF API produces random weights — the model will not work.
To use this model you need the pipeline code which defines QuantizedLinear and QuantizedMoeExperts classes. A proper load_quantized_model() function is not yet implemented — this is a research artifact, not a production-ready quantized model.
Not compatible with: llama.cpp, GGUF, vLLM, SGLang, Ollama, MLX, or any standard inference engine.
For production inference use lmstudio-community/Qwen3.6-35B-A3B-GGUF (Q4_K_M = 21.2 GB, works everywhere).
Scales salient weight channels based on activation magnitudes. Scales fused into RMSNorm weights.
Rotates weight matrices with random Hadamard matrix to remove outlier features. RMSNorm weights fused into adjacent linear layers before rotation (required for rotation equivariance). Final RMSNorm fused into lm_head.
Per-expert sensitivity analysis → 3-bit for less sensitive experts, 4-bit for sensitive ones. Group quantization (group_size=64) with scale + zero-point.
Corrects systematic quantization bias by comparing FP16 vs quantized layer outputs on calibration data.
7 bugs found and fixed during development:
| # | Bug | Effect | Fix |
|---|---|---|---|
| 1 | GatedDeltaNet wrong projection names | 30/40 layers unrotated | Fixed: in_proj_qkv/z/b/a + out_proj |
| 2 | MoE gate/router not rotated | Routing broken | Added gate + shared_expert_gate rotation |
| 3 | --skip-deltanet-rotation default=True | 30/40 layers skipped | Default=False |
| 4 | Per-layer hooks caused double rotation |
Key insight: Qwen3.6 RMSNorm uses output = x_norm * (1 + weight), not x_norm * weight. Weight init=zeros, effective multiplier = 1+w. Fusion must multiply by (1+w) and set weight to zeros after.
| Run | Config | PPL | Delta | Size |
|---|---|---|---|---|
| run1 | QuaRot only (pre-fix) | 71893 | +253246% | 17.9 GB |
| run2 | QuaRot + 3/4-bit (pre-norm-fix) | 45.8 | +61.5% | 17.9 GB |
| run3 | QuaRot + 3/4-bit (all fixes) | 30.89 | +8.87% | 17.9 GB |
| Test | Status | Notes |
|---|---|---|
| PPL (50 samples, 682 tokens) | ✓ | +0.03% vs FP16 (in-memory) |
| QuaRot rotation lossless | ✓ | -0.11% without quantization |
| HumanEval | ✗ | Not run |
| MMLU | ✗ | Not run |
| HellaSwag | ✗ | Not run |
| ARC | ✗ |
Limitations:
load_quantized_model() function not implementedmodel-*.safetensors — quantized weights (custom format)config.json — model config (quantization_config: custom)quantization_meta.json — per-layer quantization metadatazero_loss_pipeline_cuda.py — pipeline code (QuantizedLinear, QuantizedMoeExperts)METHODS_LOG.md — full development historyRESULTS.md — run results tableTongyi Qianwen (same as base model).
| Residual stream rotated N times |
| Removed hooks, rotate embed+lm_head instead |
| 5 | AWQ gate not scaled | Routing broken | Added gate + shared_expert_gate scaling |
| 6 | RMSNorm fusion used w instead of (1+w) | x2 multiplier instead of (1+w) | Multiply by (1+w), set weight to zeros |
| 7 | Final RMSNorm not fused into lm_head | +47% PPL (norm not rotation-equivariant) | Fuse final norm weight into lm_head before rotation |
| run4 |
| AWQ + QuaRot + bias (30% 4-bit) |
| 30.04 |
| +5.86% |
| 17.9 GB |
| run5 | AWQ + QuaRot + bias (50% 4-bit) | 29.64 | +4.45% | 18.5 GB |
| run6 | All 4-bit, gs=64 | 28.23 | -0.53% | 21.2 GB |
| run7 | 50% 4-bit, gs=64 | 28.98 | +2.12% | 19.6 GB |
| run8 | 60% 4-bit, gs=64 | 28.39 | +0.03% | 19.94 GB |
| Not run |
| GSM8K | ✗ | Not run |
| Long context | ✗ | Not tested |