GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
kkuspa
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
kkuspa
Model Tree
DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP
Input Modalities
Output Modalities
Supported Functionality
bash
vllm serve kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16 \--max-model-len 32768 --gpu-memory-utilization 0.85
Tested on vLLM 0.26.0. The server selects the Marlin NVFP4 kernel and warns that the GPU lacks native FP4 compute. The warning is expected: activations are BF16 by design, so the FP4 tensor-core path does not apply.
For faster decoding, enable the built-in MTP drafter:
bash
vllm serve kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16 --max-model-len 32768 --gpu-memory-utilization 0.85 --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":5}'
DavidAU tuned this model for specific sampler settings. They translate to the OpenAI API as follows:
python
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")# Thinking mode (default template): creative work, reasoningr = client.chat.completions.create(model="kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16",messages=[{"role": "user", "content": "Open a noir story set in a lighthouse."}],temperature=1.0, top_p=0.95, max_tokens=2048,extra_body={"top_k": 20, "min_p": 0.0, "repetition_penalty": 1.0},)# Coding: temperature=0.6, same top_p and top_k# Instruct mode: temperature=0.7, top_p=0.80, presence_penalty=1.5
The repo ships two chat templates. chat_template.jinja (thinking) is the default. For instruct-style responses without a thinking block, pass --chat-template pointing at chat_template-instruct.jinja at serve time.
At the time of publication this is the only quant of Fable-Fusion 711 that stock vLLM can serve. The other safetensors quants of this model use a custom runtime format.
Measured with lm-evaluation-harness 0.4.12 (vLLM backend, 0-shot, batch auto) against the BF16 parent on the same hardware:
| Task | BF16 | NVFP4A16 | Delta |
|---|---|---|---|
| ARC-Challenge (acc_norm) | 0.6126 | 0.6203 | +0.008 |
| HellaSwag (acc_norm) | 0.8502 | 0.8490 | -0.001 |
| Winogrande (acc) | 0.7806 | 0.7814 | +0.001 |
Deltas sit inside the standard error on every task (ARC-Challenge stderr is 0.014). The quantization is a data-free RTN pass. No calibration dataset was used, so nothing was tuned toward any benchmark. Absolute scores depend on harness settings; the point of this table is the like-for-like comparison, both configs measured 0-shot on the same harness and hardware.
Single request, 512-token greedy generations, 1x RTX PRO 6000 Blackwell (96 GB), vLLM 0.26.0:
| Config | Decode speed |
|---|---|
| Standard decoding | 56.2 tok/s |
| MTP speculative, depth 5 | 87.4 tok/s (1.56x) |
Acceptance across the test: 996 of 3000 drafted tokens (about 1.7 extra tokens per draft window). Throughput under concurrent load will differ.
llm-compressor 0.12.0, one-shot:
python
QuantizationModifier(targets="Linear",scheme="NVFP4A16",ignore=["lm_head", "re:.*visual.*", "re:.*linear_attn.*", "re:.*mtp.*"],)
Only the Linear weights in the full-attention and MLP blocks carry FP4. The DeltaNet linear-attention layers, the vision tower, the MTP drafter, and lm_head keep BF16, because hybrid-attention layers and drafters degrade badly under 4-bit quantization. The exact recipe ships in recipe.yaml.
preprocessor_config.json and video_preprocessor_config.json here come from Qwen/Qwen3.6-27B.model-mtp-bf16.safetensors and are wired into model.safetensors.index.json. Transformers drops them on load; they were re-extracted from the parent checkpoint so speculative decoding remains possible.The parent model is refusal-ablated ("Uncensored", "Heretic") and tuned for unconstrained creative writing. This quant changes none of that. It will produce content that aligned models refuse. You are responsible for how you deploy it.
Qwen/Qwen3.6-27B by the Qwen team, then the Fable-Fusion 711 multi-stage tune and merge by DavidAU with collaborators credited on the parent card, then this quantization. Thanks to all of them. Apache-2.0 throughout.