GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Joestar79
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Joestar79
Model Tree
OBLITERATUS/Qwen3.8-27B-OBLITERATED
Input Modalities
Output Modalities
Supported Functionality
model.safetensors (19.7 GB): FP4-packed weights, deployment tensor naming
(weight_scale / weight_scale_2), vision tower in BF16model_mtp.safetensors (1.7 GB): the 15 BF16 MTP tensors, mapped through the
safetensors index (vLLM convention)quantization_config in config.json (compressed-tensors
config_groups + quant_algo: W4A16_NVFP4 + quant_method: modelopt)qwen3_coder)Excluded from quantization (kept BF16): vision tower, MTP head, lm_head, embeddings,
and the conv1d / in_proj_a / in_proj_b internals of the linear-attention layers.
The fused in_proj_qkv / in_proj_z projections are intentionally quantized: on
this architecture they must ship packed or vLLM's fused-module loader rejects them.
bash
vllm serve /path/to/model \--served-model-name qwen3.8-27b-obl \--max-model-len 126976 \--gpu-memory-utilization 0.94 \--kv-cache-dtype fp8_e4m3 \--max-num-seqs 8 \--speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":4}' \--mamba-cache-mode align \--reasoning-parser qwen3 \--enable-auto-tool-choice --tool-call-parser qwen3_coder
Notes:
--speculative-config and keep the concurrency.qwen3_coder is the correct tool parser for this checkpoint's template
(<tool_call><function=NAME>), not qwen3_xml.| Load | Throughput |
|---|---|
| Single stream, MTP ns=4 | ~40 tok/s |
| 4 concurrent streams | 161 tok/s aggregate |
| 8 concurrent streams | 317 tok/s aggregate (~40 per stream) |
| GPU KV cache | 191,118 tokens |
| Warm startup | ~140 s |
This is an abliterated ("uncensored") derivative: refusal behavior has been removed upstream. It will comply with requests the base model would decline. You are responsible for the legality and consequences of what you generate, and for adding whatever guardrails your deployment requires. Do not expose it unauthenticated to the public internet.
Quantization was not calibrated per task; validate against the BF16 upstream for task-critical accuracy. The base model's knowledge, biases, and failure modes carry over.
Qwen/Qwen3.8-27B (Apache-2.0) -> OBLITERATUS V3 abliteration -> ModelOpt 0.45.0
W4A16 NVFP4 (this repo). Full build story and pitfall list:
Three Days, Four Wrong Hypotheses, and One Uncensored 27B That Finally Serves on a Single RTX 5090
Thanks to OBLITERATUS for the careful abliteration work, to hwkranger's Qwen3.8-27B-heretic-ara-NVFP4 whose public tensor layout was the blueprint for a loadable checkpoint, and to the Unsloth team for pushing NVFP4 on consumer Blackwell.