GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
windowsxp811203
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
windowsxp811203
Model Tree
windowsxp811203/Qwen3.8-27B-Abliterated
Input Modalities
Output Modalities
Supported Functionality
| group | treatment | count |
|---|---|---|
MLP gate/up/down (64 layers) + full-attention q/k/v/o (16 layers) | NVFP4 — 4-bit float, group size 16, float8_e4m3 scales | 256 Linears |
mtp.* (draft head) | bf16, grafted back after quantization, in ignore | 15 tensors |
model.visual.* (vision tower) | bf16 — kept bit-identical (as in essentially every other NVFP4 build of this model) | 167 weight tensors (333 incl. biases/norms) |
linear_attn.* (Gated DeltaNet / SSM) | bf16, all of it. This is the one place these builds differ from most: 18 of 87 surveyed artifacts leave every projection in the path unquantized, the rest quantize 14–71 % of those modules. (AMD's official Quark build of the 2.4T model excludes *linear_attn* outright too.) | 336 weight tensors; 432 keys incl. A_log/dt_bias |
lm_head, embeddings | bf16 |
The MTP tensors are both grafted back and listed in quantization_config.ignore. Both halves
matter: without the graft there is no draft head at all, and without the ignore entry vLLM's
compressed-tensors loader treats the bf16 head as a quantization target, finds no scales, and
rejects every draft — 0 % acceptance while the logs look perfectly clean.
Measured on this exact checkpoint on an RTX PRO 6000 Blackwell.
MTP speculative decoding — {"method":"mtp","num_speculative_tokens":1}:
| metric | value |
|---|---|
| Avg draft acceptance rate | 76.4 % – 78.3 % |
| Mean acceptance length | 1.76 – 1.78 |
| Accepted / drafted | 1615 / 2113 tokens |
That number is the proof the graft worked; a broken MTP head reads 0 %.
Refusal — greedy, non-thinking, no prefill jailbreak. For scale: unmodified Qwen3.8-27B refuses 99.04 % of the full 520-prompt AdvBench (515/520) under these settings; this build was evaluated on an 80-prompt subset, so the two denominators differ:
| benchmark | result |
|---|---|
| AdvBench (80-prompt subset) | 0/80 · 0.00 % |
| HarmBench safety categories (119) | 0/119 · 0.0 % |
| HarmBench copyright (41) | 17/41 · 41.5 % |
Safety categories = chemical/biological, cybercrime, harassment, harmful, illegal, misinformation
— every one exactly zero. The copyright column is not a safety refusal and is mostly
classifier false positives: the model delivers the lyrics or passage, but the text trips the
keyword list (either the generated prose itself opens with "I cannot quite…", or a pedantic
"I cannot generate a new passage … but here is a long excerpt" precedes the excerpt).
Capability — MMLU, 400 equidistant questions, identical prompting and parsing for both:
| build | MMLU |
|---|---|
| GGUF Q8_0 (reference) | 78.00 % |
| NVFP4A16 (this) | 77.75 % |
One question apart. (Do not compare these to the parent card's 82.35 %: that figure was measured by next-token logit comparison, a different and more forgiving method. Only same-method numbers are comparable.)
bash
vllm serve windowsxp811203/Qwen3.8-27B-Abliterated-NVFP4 \--max-model-len 8192 \--speculative-config '{"method":"mtp","num_speculative_tokens":1}'
Thinking is on by default; disable per request with
"chat_template_kwargs": {"enable_thinking": false}.
The model's declared native limit is 262,144. The 1M-window configuration from the official Qwen3.8-27B recipe loads and serves on this checkpoint; the longest prompt actually measured through it was 823,878 tokens:
bash
vllm serve windowsxp811203/Qwen3.8-27B-Abliterated-NVFP4 \--tensor-parallel-size 2 --max-model-len 1010000 \--hf-overrides '{"text_config": {"max_position_embeddings": 1010000}}'
Needle-in-a-haystack, passphrase buried at 50 % depth, greedy:
| context | prompt tokens | retrieved | time |
|---|---|---|---|
| 1M window / 824K prompt | 823,878 | ✅ | 364 s on 2× RTX PRO 6000 Blackwell |
Budget ≈ 61 GiB of KV at 1M (16 full-attention layers × 4 KV heads × 256 dim) on top of the 26.6 GiB of weights, so 1M needs two 96 GB cards; 256K fits comfortably on one.
On hosts where the CUDA toolkit and FlashInfer's bundled CCCL headers disagree, FlashInfer's JIT
fails to build its sampling kernels and vLLM aborts with
FlashInfer requires GPUs with sm75 or higher (a misleading message — the real cause is that the
capability probe itself failed). Working around it:
bash
export VLLM_USE_FLASHINFER_SAMPLER=0export VLLM_ATTENTION_BACKEND=TRITON_ATTN
This is a host toolchain issue, not a property of these weights.
Quantized with llm-compressor 0.13.0 (NVFP4A16) from the bf16 parent, which was produced by
orthogonalizing 131 residual-writing tensors (including embed_tokens) against a refusal direction
at λ=1.5, leaving the vision tower byte-identical. Full recipe and evaluation in the
parent model card.
A llama.cpp build is at
Qwen3.8-27B-Abliterated-GGUF.
If these models are useful to you, tips are appreciated — they pay for the GPU time. 如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。
USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888
This model will not refuse. It is published for alignment and safety research. You are responsible for your use of it and for complying with applicable law. Inherits the Apache-2.0 license of the base model.