GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
groxaxo
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
groxaxo
Model Tree
Input Modalities
Output Modalities
Supported Functionality
| Parameter | Value |
|---|---|
| Algorithm | GPTQ-Pro (gptqmodel) |
| Bits / group size | 4 / 64 |
| Preset | quality (QuantizeConfig.quality_4bit) |
| Calibration | text, 64 samples ≈ 315k tokens, avg ~19.7k chars/sample |
| Calibration device | CPU (--calib-device cpu) |
| Offload | disk (--offload-disk) |
| Source precision | BF16 (18 safetensors shards) |
model_type=qwen3_5, Qwen3_5ForConditionalGeneration — Qwen3.8-27B reuses the
exact 64-layer/400-linear contract of Qwen3.6/3.5-27B.Structural checks (bits=4, g=64, 64 layers intact, tokenizer present, shard sizes),
CPU load + deterministic 24-token generation, degenerate-output detection.
Full log: qwen3_8_27b_preflight.json in this repo.
--preset max_quality, --nsample 128, ~2x runtime,
8 h budget) is being quantized next and will be published separately as
…-GPTQ-Pro-4bit-g64-maxq when its quality gate passes.python
from transformers import AutoTokenizer, AutoModelForCausalLMmodel_id = "groxaxo/Huihui-Qwen3.8-27B-abliterated-GPTQ-Pro-4bit-g64"tok = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
GPTQ-Pro inference requires gptqmodel (or any GPTQ kernel that reads
quantization_config); vLLM ≥ current stable also loads these checkpoints.
Apache-2.0 (inherited from the base model). Abliteration disclaimer: this build has safety-refusal conditioning removed by the upstream Huihui project; use responsibly and in accordance with your local regulations.