Model Details
Table | |
|---|
| Developed by | cbrigance |
| Base model | Qwen/Qwen3.8-27B (Apache-2.0) |
| Model type | Causal language model with vision encoder (native VLM) |
| Architecture | Qwen3_5ForConditionalGeneration (model_type: qwen3_5) |
| Modalities | Text in / text out; image and video understanding |
| Languages | Multilingual (inherited from base); abliteration tuned with an English system prompt |
| Parameters | 27,356,728,560 (27.36 B) |
| Precision | BF16, unquantized |
| On-disk size | 54.7 GB across 12 safetensors shards |
| Context length | 262,144 tokens native (base is documented as extensible to ~1M) |
| License | Apache-2.0, inherited from the base model |
| Modification method | Directional ablation (abliteration) — no gradient training, no fine-tuning data |
Architecture summary
Inherited unchanged from Qwen3.8-27B:
- Language model: 64 layers, hidden dim 5120, FFN intermediate dim 17,408
- Hidden layout: 16 × ( 3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN) )
— i.e. linear attention in 48 layers, full attention in 16 (every 4th layer)
- Gated DeltaNet: 48 value heads / 16 QK heads, head dim 128, conv kernel dim 4
- Gated Attention: 24 Q heads / 4 KV heads (GQA), head dim 256, partial rotary factor 0.25 (RoPE dim 64),
rope_theta 10,000,000, interleaved M-RoPE
- Vocabulary: 248,320 (padded), untied input/output embeddings
— embeddings account for 2,542,796,800 params, leaving ≈24.8 B non-embedding
- Vision tower: 27-block ViT, hidden 1152, intermediate 4304, patch size 16, spatial merge 2, temporal patch 2, output projected to 5120 (≈0.41 B params in the blocks)
- Thinking: on by default, with
reasoning_effort ∈ {low, medium, xhigh} and preserve_thinking
How it was made
Heretic performs directional ablation: it
identifies the latent direction most associated with refusal by contrasting
activations on harmful vs. harmless prompts, then projects that direction out of
selected weight matrices. It uses TPE-based hyperparameter search to trade off
refusal suppression against KL divergence from the original model. No weights
are trained — the transformation is a closed-form projection, so no new data is
memorized into the model.
Search configuration
Table with columns: Setting, Value| Setting | Value |
|---|
| Harmful prompts (direction finding) | mlabonne/harmful_behaviors, train — 416 prompts (entire split) |
| Harmless prompts (direction finding) | mlabonne/harmless_alpaca, train — 800 prompts |
| Refusal eval set | mlabonne/harmful_behaviors, test[:100] (held out) |
| KL reference set | mlabonne/harmless_alpaca, test[:100] (held out) |
| System prompt during search | "You are a helpful and abliterated English-speaking assistant who thinks/reasons as little as possible." |
| Max response length | 300 tokens |
| Batch size | 128 |
| Compute dtype | bfloat16 |
Refusals were detected by substring matching against an extended marker list
tuned for Qwen-family phrasing ("can't assist", "not appropriate",
"feel free to ask", "different topic", "consulting", "proper authorities", …).
Heretic's common-prefix check found no shared response prefix to normalize away,
so refusal counts are taken over raw completions.
Run environment
Table with columns: Setting, Value| Setting | Value |
|---|
| Heretic version | v1.4.0 |
| Hardware | 1 × NVIDIA GB10 (121.63 GB VRAM) |
| CUDA / driver | 13.3 / 580.173.02 |
| Trials | 800 (TPE search) |
| Wall-clock | 53 h 15 m |
| Peak VRAM | 51.1 GB allocated / 73.0 GB reserved |
The reserved figure is the number to plan against if you want to repeat this: a
27 B BF16 abliteration search needs roughly 73 GB, not the ~55 GB the weights
alone suggest.
Selected trial
Trial 743 was selected from the search:
Table with columns: Metric, Base model, Trial 743| Metric | Base model | Trial 743 |
|---|
| Refusals on held-out eval | 89 / 100 | 28 / 100 |
| KL divergence from base | — | 0.0409 |
Refusal markers fell from 89/100 to 28/100 — a 68.5 % reduction — on the same
held-out eval set. Read that as a drop in refusal phrasing, not a clean
measurement of safety behavior: detection is substring-based, so a compliant but
useless answer scores as a non-refusal in both columns.
A KL of 0.0409 on harmless prompts is low, indicating the model's behavior on
benign inputs remains close to the base — but see
Limitations for why that is not a capability
guarantee.
Ablation parameters
direction_index = 34.55 attn.o_proj.max_weight = 1.20attn.o_proj.max_weight_position = 42.46attn.o_proj.min_weight = 0.39attn.o_proj.min_weight_distance = 30.92 mlp.down_proj.max_weight = 1.49mlp.down_proj.max_weight_position = 38.94mlp.down_proj.min_weight = 1.42mlp.down_proj.min_weight_distance = 30.26
The projection is applied to attention output projections (o_proj) and MLP
down-projections (down_proj) across the language model, weighted by a
position-dependent envelope centered near layers 38–42.
Scope — 128 modules, all in the language model:
Table with columns: Component, Modules ablated| Component | Modules ablated |
|---|
attn.o_proj | 64 (every layer) |
mlp.down_proj | 64 (every layer) |
| Vision tower | 0 |
Two details this makes explicit. First, the vision tower was not modified —
Heretic enumerated no abliterable modules inside it. Second, attn.o_proj
matched all 64 layers, not only the 16 full-attention ones, so the Gated
DeltaNet output projections were ablated as well. Despite the 48/16
linear/full attention split described above, every layer in the stack was
touched.
direction_index = 34.55 selects a single global refusal direction, fractionally
interpolated between the per-layer candidate directions — here roughly layer
34–35 of 64. Heretic can instead give each layer its own direction; see below for
how that fared.
Search landscape
The selected trial is not the last one run. Trial 800, the final trial, used
per-layer directions rather than one global direction:
Table with columns: Trial, Direction, Refusals, KL| Trial | Direction | Refusals | KL |
|---|
| 743 (selected) | global, index 34.55 | 28 / 100 | 0.0409 |
| 800 (final) | per layer | 85 / 100 | 0.0680 |
Per-layer directions were worse on both axes — barely below the 89/100 baseline
while drifting further from the base model. The objective surface is rough, and
good configurations are found rather than converged upon; treat the published
hyperparameters as one sampled point, not an optimum.
Differences from the base checkpoint
Beyond the weight ablation itself, this checkpoint differs from
Qwen/Qwen3.8-27B in ways worth knowing before you deploy or quantize it:
- The MTP (multi-token prediction) head was dropped. The base repo ships 15
mtp.* tensors; this checkpoint has 0 (1184 tensors vs. the base's 1199).
config.json still declares mtp_num_hidden_layers: 1, but the weights are
absent. MTP-based speculative decoding will not work with this checkpoint
or anything quantized from it. Use a draft model or n-gram speculation instead
if you need speculative decoding.
This is not a Heretic decision: the run log shows 1184 tensors at load time,
before any ablation. Qwen3_5ForConditionalGeneration under Transformers
5.14.1 never materializes the MTP head, so it is dropped by the load/save
round-trip — any Transformers-based re-serialization of the base model will
lose it the same way.
- Re-serialized by Transformers 5.14.1 (base was written by 5.8.0.dev0).
config.json gained explicit dtype: bfloat16 fields and the vision config's
model_type is now qwen3_5_vision.
- Preprocessor files are consolidated. This repo carries a single
processor_config.json (Transformers v5 style) rather than the base's separate
preprocessor_config.json + video_preprocessor_config.json. Image/video
preprocessing parameters are unchanged.
- Legacy tokenizer files are absent.
vocab.json and merges.txt are not
included; tokenizer.json is complete and is what modern loaders use. Tools
that require the legacy pair (some GGUF conversion paths) may need them copied
from the base repo.
Files
config.json chat_template.jinjageneration_config.json tokenizer.jsonprocessor_config.json tokenizer_config.jsonmodel.safetensors.index.json model-0000{1..12}-of-00012.safetensorsHERETIC.md README.md
HERETIC.md contains the raw trial output from the abliteration run.
Usage
This card covers the BF16 checkpoint, intended as the source for
quantization and for direct transformers use. Serving instructions for
production engines live on the quantized model cards.
Requirements
pip install -U "transformers>=5.14.1" accelerate torchvision
BF16 inference needs ≥55 GB of VRAM for weights alone, plus KV cache. Plan on
2×80 GB (or 4×48 GB with device_map="auto") for meaningful context lengths.
Text
import torchfrom transformers import AutoModelForImageTextToText, AutoProcessor model_id = "cbrigance/Qwen3.8-27B-heretic" processor = AutoProcessor.from_pretrained(model_id)model = AutoModelForImageTextToText.from_pretrained( model_id, dtype=torch.bfloat16, device_map="auto",) messages = [{"role": "user", "content": "Explain the Gated DeltaNet attention mechanism."}] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", reasoning_effort="xhigh", # "low" | "medium" | "xhigh" (default)).to(model.device) out = model.generate(**inputs, max_new_tokens=2048, temperature=1.0, top_p=0.95, top_k=20)print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Image
messages = [{ "role": "user", "content": [ {"type": "image", "url": "https://example.com/diagram.png"}, {"type": "text", "text": "What does this diagram show?"}, ],}] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",).to(model.device) out = model.generate(**inputs, max_new_tokens=1024)print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Videos work the same way with {"type": "video", "url": ...}. Note the chat
template raises an exception if a system message contains images or video —
put visual content in user turns only.
Thinking control
Thinking is on by default; the template emits <think>\n after the
generation prompt.
Table with columns: Template kwarg, Effect| Template kwarg | Effect |
|---|
enable_thinking=False | Emits a closed empty <think></think> block — direct answer, no reasoning |
reasoning_effort="low" | "medium" | "xhigh" | Injects a system-level reasoning-depth instruction. Default xhigh. Any other value raises |
preserve_thinking=False | Keeps reasoning traces only from the latest user turn instead of all history |
Sampling parameters
Inherited from the base model's recommendations:
- Thinking mode:
temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
- Non-thinking mode:
temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
generation_config.json ships do_sample=true, temperature=1.0, top_p=0.95,
top_k=20, EOS [248046, 248044] (<|im_end|>, <|endoftext|>).
Note that the abliteration search used a system prompt asking the model to
"think/reason as little as possible." The selected direction was optimized
under that condition, so refusal suppression may be somewhat less consistent at
reasoning_effort="xhigh" with long deliberation than at low.
The base model's XML tool-call format is preserved unchanged:
<tool_call><function=function_name><parameter=param_name>value</parameter></function></tool_call>
Passing tools=[...] to apply_chat_template injects the tool schema and format
instructions into the system message automatically.
Quantized versions
Derived quantizations are published separately, each with its own card and
engine-specific launch instructions:
Table with columns: Format, Target engine, Status| Format | Target engine | Status |
|---|
| GGUF | llama.cpp / Ollama | planned |
| NVFP4 | vLLM (Blackwell) | planned |
| FP8 | vLLM (Hopper+) | planned |
If you are quantizing this checkpoint yourself: the vision tower and the
linear_attn state-space parameters (A_log, dt_bias, conv1d) should be
kept at higher precision. mamba_ssm_dtype is float32 in the config for a
reason, and aggressive quantization of SSM state parameters degrades long-context
behavior disproportionately.
Evaluation
Code Generation Benchmarks
Benchmarks run using EleutherAI lm_eval v0.4.13
on an NVIDIA GB10 (121.63 GB unified VRAM/RAM). Model loaded in float16 with
apply_chat_template=True and enable_thinking=False.
Table with columns: Benchmark, Ablated, Original, Delta| Benchmark | Ablated | Original | Delta |
|---|
| HumanEval (pass@1) | 95.1% | 96.3% | −1.2% |
| MBPP (pass@1) | 74.8% | 76.0% | −1.2% |
Minimal degradation: both benchmarks show only −1.2% regression, well within
the range seen across the 12-model ablation study.
Broader capability caveats
The base model's published benchmark scores on other tasks are not transferable to this
checkpoint and are deliberately not reproduced here. Abliteration is known to
cause measurable capability regression — commonly in instruction-following
precision, calibration, and refusal of genuinely impossible or nonsensical
requests. A low KL on short harmless completions does not rule out degradation on
long-horizon agentic tasks, multi-step reasoning, or multimodal grounding, none
of which the search objective measured.
If you benchmark this model, please open a discussion with your results.
Bias, Risks, and Limitations
Safety behavior has been intentionally removed
Abliteration suppresses the model's learned refusal direction. This checkpoint
will produce content on topics the base model declines, including content that is
dangerous, illegal, or harmful. This is the explicit purpose of the modification,
not a defect.
Abliteration degrades refusal behavior; it does not eliminate it. 28 of 100
held-out harmful prompts still triggered refusal markers, down from 89 of 100 in
the base model — a large reduction, but nowhere near zero. Behavior is
inconsistent — the same request may be answered or refused across runs, system
prompts, reasoning-effort settings, or languages. Do not treat this model as
either reliably safe or reliably compliant. Neither property holds.
Conversely, refusal suppression is not selective. The mechanism removes a general
direction, so the model may also lose appropriate hedging on medical, legal, and
financial questions — confidently asserting where the base model would have
recommended consulting a professional. Note that markers like "recommend",
"consulting", "medical professional", and "seek help" were explicitly
penalized as refusals during the search, so this loss of hedging is directly
selected for, not incidental.
Inherited limitations
All base-model limitations carry over: hallucination, training-data cutoff,
cultural and linguistic bias in pretraining data, and imperfect visual grounding
(OCR errors, spatial-reasoning failures, miscounting). The vision tower was not
modified, so multimodal behavior is nominally unchanged — but it has not been
verified, and vision inputs are a plausible vector for eliciting behavior the
text-side ablation was never evaluated against.
Technical limitations
- MTP speculative decoding is unavailable (head not present in this checkpoint)
- BF16 at 54.7 GB is impractical for single-GPU deployment; use a quantized release
- The 262K context is inherited but untested post-ablation; long-context quality has not been measured
- Refusal-marker matching is substring-based, so the 28/100 figure counts phrasing, not semantics — a compliant-but-useless answer scores as non-refusal
Out-of-scope use
This model should not be used for:
- Any deployment serving minors or vulnerable populations
- User-facing products without an independent moderation layer on both input and output
- Medical, legal, financial, or psychological advice presented as authoritative
- Automated decisions affecting people's rights, safety, employment, or access to services
- Generating content that is illegal in your jurisdiction, targets real people for harassment, or is used to produce CSAM, non-consensual intimate imagery, or targeted disinformation
- Any use prohibited by the Apache-2.0 license or applicable law
Recommendations
- Run external safety filtering. Treat this model as having no built-in guardrails and apply classification on prompts and completions independently — e.g. Llama Guard, ShieldGemma, or a commercial moderation API.
- Keep it in controlled environments. Research, red-teaming, evaluation of alignment methods, and personal use with informed consent are the intended contexts.
- Log and review. If exposed to others, retain transcripts and audit them.
- Validate before relying on it. Benchmark against the base model on tasks you actually care about — do not assume parity from the KL figure.
Users are responsible for what they generate and for compliance with applicable
law. Neither the base-model authors nor the Heretic authors endorse this
derivative.
License and attribution
Released under Apache-2.0, inherited from Qwen/Qwen3.8-27B. The base
model's LICENSE
applies to these weights.
The directional-ablation technique originates from Arditi et al., "Refusal in
Language Models Is Mediated by a Single Direction"
(arXiv:2406.11717).
Citation
@misc{qwen3.8-27b-heretic, title = {Qwen3.8-27B-heretic}, author = {cbrigance}, year = {2026}, note = {Abliterated derivative of Qwen/Qwen3.8-27B produced with Heretic}, url = {https://huggingface.co/cbrigance/Qwen3.8-27B-heretic}} @misc{arditi2024refusal, title = {Refusal in Language Models Is Mediated by a Single Direction}, author = {Andy Arditi and Oscar Obeso and Aaquib Syed and Daniel Paleka and Nina Panickssery and Wes Gurnee and Neel Nanda}, year = {2024}, eprint = {2406.11717}, archivePrefix = {arXiv}, primaryClass = {cs.LG}}
Changelog
- 2026-08-21 — Initial BF16 release. Heretic v1.4.0, trial 743 of 800: refusals 89/100 → 28/100, KL 0.0409.