GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
gjtgjt
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
gjtgjt
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
Heretic applies parameterized directional ablation (Arditi et al., 2024): it estimates a refusal direction from first-token residuals, then orthogonalizes selected projections against that direction.
On this model the hooked modules are:
attn.o_proj on Gated Attention layers, linear_attn.out_proj on Gated DeltaNet layers (Heretic maps both to attn.o_proj)mlp.down_proj on all 64 language layersThat is 64 attention-output modules and 64 MLP modules. Embeddings, norms, other attention/MLP projections, the vision encoder, and the MTP block are left as in the base checkpoint.
TPE search (200 trials, 60 startup) jointly minimized English keyword refusals on mlabonne/harmful_behaviors and KL divergence on mlabonne/harmless_alpaca. Trial 145 (Pareto index 0, lowest keyword count) was merged to a full BF16 checkpoint.
Qwen3.8 thinks by default. apply_chat_template(..., add_generation_prompt=True) starts the assistant turn with an open <think>\n. Official instruct / non-thinking mode closes that as:
text
<think></think>
Heretic’s default CoT skip looks for exact <think></think>, which this model does not emit. Residuals and KL collected at the open <think> token are dominated by the thinking opener, not by the answer distribution.
Search and evaluation therefore used:
bash
--response-prefix $'\n</think>\n\n'
This is only a measurement cut. The shipped chat_template.jinja is the official one. At inference time, use enable_thinking=False for instruct mode; do not bake the Heretic prefix into the template.
| Parameter | Value |
|---|---|
direction_index | per layer |
attn.o_proj.max_weight | 0.88 |
attn.o_proj.max_weight_position | 57.48 |
attn.o_proj.min_weight | 0.26 |
attn.o_proj.min_weight_distance | 19.43 |
mlp.down_proj.max_weight |
bash
PREFIX=$'\n</think>\n\n'heretic \--model Qwen/Qwen3.8-27B \--response-prefix "$PREFIX" \--n-trials 200 \--n-startup-trials 60 \--winsorization-quantile 1.0 \--model-action save \--export-strategy merge \--save-directory Qwen3.8-27B-hereticheretic \--model Qwen/Qwen3.8-27B \--evaluate-model Qwen3.8-27B-heretic \--response-prefix "$PREFIX"
Same instruct cut as above. Keyword markers and both prompt sets are English.
| Metric | Base | This model |
|---|---|---|
| Keywords / 100 harmful prompts | 98 | 27 |
| KL / 100 harmless prompts | 0 | 0.0446 |
Sanity checks on the merged weights (greedy, short decode): instruct 1+1 answers 2 directly; a Fibonacci request yields a normal Python function; thinking mode still emits <think>…</think> then answers.
Abliteration lowers templated English refusals. It is not a safety guarantee. Chinese refusal rate and vision/MTP quality were not measured. Thinking-mode refusal behavior can differ from the instruct-cut numbers.
python
from transformers import AutoModelForImageTextToText, AutoTokenizermodel_id = "Qwen3.8-27B-heretic" # replace with the Hub repo id after uploadtokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForImageTextToText.from_pretrained(model_id,dtype="auto",device_map="auto",)messages = [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Explain residual connections in one paragraph."},]# Instruct / non-thinking (the Heretic evaluation cut)text = tokenizer.apply_chat_template(messages,add_generation_prompt=True,tokenize=False,enable_thinking=False,)# Thinking mode (default): omit enable_thinking, or set True
Recommended sampling follows the base card:
temperature=1.0, top_p=0.95, top_k=20temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5Full BF16 needs about 51 GiB of model weights, plus KV cache. For serving, the same stacks as the base model apply (Transformers, vLLM, SGLang, and so on). For architecture details, long-context YaRN, and multimodal serving, see Qwen/Qwen3.8-27B.
Apache-2.0, inherited from the base model.
bibtex
@misc{qwen38,title = {{Qwen3.8-Max}: A New Bar for Coding and Cowork},url = {https://qwen.ai/blog?id=qwen3.8},author = {{Qwen Team}},month = {August},year = {2026}}@misc{heretic,author = {Weidmann, Philipp Emanuel},title = {Heretic: Fully automatic censorship removal for language models},year = {2025},publisher = {GitHub},howpublished = {\url{https://github.com/p-e-w/heretic}}}
官方 Qwen/Qwen3.8-27B 的 Heretic v1.4.0 消融版。不是新预训练，也不是 SFT。只改语言层一部分投影权重；分词器、对话模板、视觉塔和 MTP 与底模相同。
| 项 | 值 |
|---|---|
| 底模 | Qwen/Qwen3.8-27B，revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 |
| 架构 | Qwen3_5ForConditionalGeneration，64 层稠密混合：48×Gated DeltaNet + 16×Gated Attention |
| 精度 | BF16，12 个分片，约 51 GB |
| 工具 | Heretic 1.4.0，transformers 5.15.0 |
Heretic 用有害 / 无害题的首 token 残差估计拒绝方向，再对选定投影做正交化。本模型挂钩的是语言层：
attn.o_proj，Gated DeltaNet 的 linear_attn.out_proj（Heretic 都映射成 attn.o_proj）mlp.down_proj视觉塔和 MTP 未消融。搜索为 200 trial / 60 startup 的 TPE，在 instruct（关思考）切点上同时压 Keyword 拒绝率和无害题 KL。导出的是 Pareto 上 keyword 最低的 trial 145。
Qwen3.8 默认开思考。apply_chat_template 会在 assistant 开头留下未闭合的 <think>\n。官方关思考空块是 <think>\n\n</think>\n\n。Heretic 默认 CoT skip 找的是精确 <think></think>，对不上。若在未闭合的 <think> 处采残差 / 算 KL，下一个 token 几乎被思考开场锁死，指标会失真。
因此搜索和复评都加了 --response-prefix $'\n</think>\n\n'。这只用于测量，成品模板仍是官方模板。推理时关思考请传 enable_thinking=False。
消融参数见上文英文表。复现命令见上文 Reproduce。
取点均为上述 instruct 切点。Keyword 词表和评估集都是英文。
| 指标 | 原版 | 本模型 |
|---|---|---|
| Keywords / 100 有害题 | 98 | 27 |
| KL / 100 无害题 | 0 | 0.0446 |
消融降低的是模板化英文拒答，不是安全保证。中文拒答、视觉和 MTP 本轮未测；开思考时的拒绝行为可能与上表不同。
用法与采样参数见英文 Usage。底模能力、长上下文 YaRN 和多模态部署见官方模型卡。许可为 Apache-2.0。
| 1.30 |
mlp.down_proj.max_weight_position | 59.74 |
mlp.down_proj.min_weight | 1.14 |
mlp.down_proj.min_weight_distance | 8.36 |