GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
maci0
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
maci0
Model Tree
Base
Jackrong/Qwopus3.6-27B-Coder
Quantized
this model
Input Modalities
Text
Image
Video
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
Near-lossless versus the bf16 source: wikitext-2 perplexity for this build is 6.73.
| Metric | Value |
|---|---|
| wikitext-2 PPL | 6.73 |
| Weights | NVFP4 W4A4, group 16 |
| Size | 18 GB vs 55.6 GB bf16 (~33%) |
| KL divergence | 0.0398 (capability preservation, lower is better) |
NVFP4 uses GPTQ error compensation, an MSE observer, and shared fused-layer scales, so the drop from bf16 is minimal.
NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag
needed. --reasoning-parser qwen3 splits the <think> block into reasoning_content;
--tool-call-parser qwen3_coder enables tool/function calling for agentic coding.
bash
vllm serve maci0/Qwopus3.6-27B-Coder-abliterated-NVFP4 \--served-model-name qwopus-27b-coder-abliterated-nvfp4 \--max-model-len 131072 \--gpu-memory-utilization 0.90 \--kv-cache-dtype fp8 \--reasoning-parser qwen3 \--enable-auto-tool-choice --tool-call-parser qwen3_coder
--max-model-len 131072 is a safe default; raise it if memory allows.--language-model-only to skip the vision tower and free KV cache for text use.python
from openai import OpenAIclient = OpenAI(base_url="http://localhost:8000/v1", api_key="x")r = client.chat.completions.create(model="qwopus-27b-coder-abliterated-nvfp4",messages=[{"role": "user", "content": "Write a Python function that merges two sorted lists."}],)print(r.choices[0].message.content)
bash
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "qwopus-27b-coder-abliterated-nvfp4","messages": [{"role": "user", "content": "Write a Python function that merges two sorted lists."}]}'
A 27B Qwen3.5-family vision-language model specialized for code (Qwopus 3.6 Coder), with thinking-mode reasoning and a 256K context window.
max_position_embeddings 262144).Heretic removes the refusal direction with a TPE-optimized
search over per-component ablation strength, jointly minimizing refusal rate and KL divergence
from the original model, then merges the best trial. This model was abliterated in two iterative
rounds: round 1 removed the dominant refusal direction, then Heretic was re-run on the round-1
model to remove the residual refusal direction that surfaced once the first was gone. Because Qwopus
is a thinking model, evaluation ran in non-thinking mode so each judged response is a real answer
rather than an unfinished <think> block; the shipped model restores the original thinking chat
template.
| Round | Refusals | Note |
|---|---|---|
| Baseline | 99/100 | original model |
| Round 1 | 25/100 | dominant refusal direction removed |
| Round 2 | 3/100 | residual direction removed (final) |
mlabonne/harmless_alpaca (good) vs mlabonne/harmful_behaviors (bad).| Scheme | NVFP4, W4A4 |
| Weight rounding | GPTQ (Hessian-based error compensation), MSE observer |
| Weights | FP4 (E2M1), group_size=16, tensor_group, FP8 (E4M3) group scales, shared across fused layers |
| Activations | FP4, dynamic per-group, FP8 (E4M3) scales |
| Quantized | all language-model Linear layers |
| Kept in bf16 | vision tower (model.visual.*), lm_head, MTP head |
GPTQ is a quantization-time cost only; inference speed and format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.
Calibration: 512 domain-matched samples (long reasoning + general chat + code),
max_seq_len=2048, text-only path through the VL model.
Thinking mode is the default.
temperature=0.6, top_p=0.95, top_k=20temperature=1.0, top_p=0.95, top_k=20temperature=0.7, top_p=0.80, top_k=20{%- set enable_thinking = false %} in the chat template, or
pass extra_body={"chat_template_kwargs": {"enable_thinking": false}}.--reasoning-parser and --tool-call-parser are not auto-detected; pass them explicitly.chat_template_kwargs.Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.
| Untouched | gated delta-net Conv1d and SSM params (A_log, dt_bias), never Linear |