GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
zrlu
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
zrlu
Model Tree
huihui-ai/Huihui-Qwen3.8-27B-abliterated
Input Modalities
Output Modalities
Supported Functionality
Apache-2.0 — inherited unchanged from the base model. See LICENSE and the
base model card. This is a weight-only quantization; no rights are widened.
gptqmodel==7.3.2; run on a CUDA GPU (RTX 5090) with lazy checkpoint
loading; calibration: wikitext-2 (or bundled fallback text).text
bits=4, group_size=128, desc_act=false, sym=true, lm_head=false, dynamic={"-:.*mtp.*":{}}Qwen3_5ForConditionalGeneration, image_token_id 248056.mtp.* tensors.Served with vLLM XPU 0.28.0 (kernels 0.1.12.3), MTP3 speculative decoding
with BF16 draft (DRAFT_INT4=0), prefix caching, qwen3_xml tool-call parser, fp8 KV,
KV cache 4.3 GiB (KV_CACHE_MEMORY_BYTES=4617089843), max-num-seqs=1.
No repetition/presence penalty — sampling is pure Qwen defaults from the
model's generation_config.json (temperature 1.0 / top_k 20 / top_p 0.95);
the deploy repo's patch_tile_mask.py boot patch covers the vLLM 0.28.0
!-degeneration bug instead. Benchmark/report details live in the
deployment repository
(benchmarks/ section).
One-click container: zrlu/qwen38-27b-arc-pro-b70:latest on Docker Hub
(entrypoint auto-downloads this repo to /model on first start; switch to any
other published B70 quant with -e HF_REPO=zrlu/<repo>), Dockerfile and
pi-agent setup in the deploy repository.
python
# standalone / evalfrom transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("zrlu/Huihui-Qwen3.8-27B-abliterated-GPTQ-Int4-sym-G128-MTP-BF16-B70")m = AutoModelForCausalLM.from_pretrained( "zrlu/Huihui-Qwen3.8-27B-abliterated-GPTQ-Int4-sym-G128-MTP-BF16-B70", device_map="auto")Signed: zrlu. GPU: Arc Pro B70 (0xe223) + RTX 5090 (quantization only).
# standalone / evalfrom transformers import AutoModelForCausalLM, AutoTokenizertok = AutoTokenizer.from_pretrained("zrlu/Huihui-Qwen3.8-27B-abliterated-GPTQ-Int4-sym-G128-MTP-BF16-B70")m = AutoModelForCausalLM.from_pretrained( "zrlu/Huihui-Qwen3.8-27B-abliterated-GPTQ-Int4-sym-G128-MTP-BF16-B70", device_map="auto")