GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
darioooooo0o
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
darioooooo0o
Model Tree
OBLITERATUS/Qwen3.8-27B-OBLITERATED
Input Modalities
Output Modalities
Supported Functionality
KL divergence over the top-20 logits on held-out 2048-token windows, against the OBLITERATED BF16 weights (lower is better):
| chat (private samples) | code | technical prose | all | top-1 agree | |
|---|---|---|---|---|---|
| this quant | 0.029 | 0.031 | 0.043 | 0.034 | 93.8% |
| this quant + fp8 lm_head (as served by the cookbook) | 0.029 | 0.032 | 0.043 | 0.034 | 93.6% |
For comparison, the same recipe on stock Qwen3.8-27B measures 0.039, and a regular symmetric GPTQ int4 measures 0.078. The abliteration survives quantization: the quant still answers requests the stock model refuses. On the B70 cookbook's 6-question arithmetic canary it scores 4/6, the same as the stock quant (that canary is mostly greedy near-ties, so it's noise).
Built for and tested on Intel Arc Pro B70 + vLLM XPU v0.31.0. Easiest is the prebuilt image
(ghcr.io/riceharvest/b70-vllm:v0.31.0-b70.3 = :latest), which has the patches and B70 attention kernels baked in:
bash
docker run -d --name b70 --device /dev/dri -v /dev/dri:/dev/dri:ro --shm-size 8g -p 8000:8000 \ -v ~/models:/models -e MODEL=darioooooo0o/Qwen3.8-27B-OBLITERATED-GPTQ-Int4-asym ghcr.io/riceharvest/b70-vllm:latestOr use the launcher from riceharvest/b70-qwen38-cookbook:
MODEL_DIR=/path/to/this/repo ./serve.sh. You get MTP k=4, fp8 KV, tool calls and up to 262K context with vision.
Measured on one B70 (stock 275 W), one request at a time, greedy, 400-token answers, at 131K max context: code ~90 tok/s, prose ~71 tok/s. Generic Qwen3.8-27B numbers on the same stack (prefill ~2,000 tok/s, ~35 ms per MTP verify step, 262K context) are in the cookbook.
Many concurrent requests: add --mamba-ssm-cache-dtype float16 (after the image name, or B70_GDN_STATE=fp16 ./serve.sh).
Qwen3.8 stores its linear-attention (GDN) state in fp32 by default, and with many requests that state is a large part of what
the GPU reads per step. Measured on Qwen3.6-35B-A3B, which has the same GDN layers: 64 concurrent users went from 49 to 42-45
ms per token, with decode quality at the fp32 noise floor and gsm8k 98/100 vs 94/100
(details). Not measured on this model yet.
Other runtimes: the format is standard asymmetric GPTQ v1, so GPTQModel / transformers / vLLM on CUDA should
load it. I only tested on Intel XPU. Stock vLLM XPU v0.31 does not load asym GPTQ correctly without the
cookbook's patch_asym_gptq.py (the prebuilt image includes it).
b70quant.py from the cookbook. GPTQ (damp 0.01, act-order off), asym uint4 with per-group
scale + zero, and a clip search over 11 ratios minimizing |err|^2.4. One layer at a time on the B70, ~55 min.in_proj_a/in_proj_b.Credits: Qwen team (base model), OBLITERATUS (abliteration), SergiioB (B70 vLLM cookbook this builds on).
docker run -d --name b70 --device /dev/dri -v /dev/dri:/dev/dri:ro --shm-size 8g -p 8000:8000 \ -v ~/models:/models -e MODEL=darioooooo0o/Qwen3.8-27B-OBLITERATED-GPTQ-Int4-asym ghcr.io/riceharvest/b70-vllm:latest