Provenance
Table with columns: Stage, Detail| Stage | Detail |
|---|
| Base model | Qwen/Qwen3.5-0.8B (0.8B VLM, hybrid linear/full attention) |
| Source checkpoint | cbrigance/Qwen3.5-0.8B-heretic (BF16 safetensors) |
| Developed by | cbrigance |
| Abliteration | Heretic v1.4.0 (self-produced, not a third-party checkpoint) |
| Quantization | NVIDIA Model Optimizer v0.45.0, W4A16 NVFP4 + FP8 KV cache |
| Hardware | HP ZGX Nano G1n (NVIDIA GB10, 128GB unified memory) |
Quantization Details
Table | |
|---|
| Format | W4A16 NVFP4 (4-bit weights, 16-bit activations), FP8 KV cache |
| Group size | 16 |
| Producer | NVIDIA Model Optimizer (modelopt) v0.45.0 |
| On-disk size | ~965 MB |
| Excluded modules | lm_head, embeddings, all linear_attn conv1d/in_proj layers, vision tower (model.visual*) |
| Requires | Blackwell-generation GPU (SM120+) for native NVFP4 support |
The vision tower and linear-attention state-space parameters are kept at higher precision.
Only the full-attention and MLP weight matrices are quantized to NVFP4.
Benchmark Evaluation (BF16 source)
Code generation benchmarks were run on the BF16 source checkpoint using
EleutherAI lm_eval v0.4.13
on an NVIDIA GB10 (121.63 GB unified VRAM/RAM). Model loaded in float16 with
apply_chat_template=True and enable_thinking=False.
Table with columns: Benchmark, Ablated, Original, Delta| Benchmark | Ablated | Original | Delta |
|---|
| HumanEval (pass@1) | 33.5% | 31.7% | +1.8% |
| MBPP (pass@1) | 22.6% | 25.0% | −2.4% |
NVFP4 quantization may introduce additional quality loss not captured here.
Quickstart
vLLM
Requires a Blackwell-generation GPU (SM120+) for native NVFP4 support.
vllm serve \ --model cbrigance/Qwen3.5-0.8B-heretic-NVFP4 \ --quantization modelopt \ --kv-cache-dtype fp8
Then query via the OpenAI-compatible API:
curl -X POST http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "cbrigance/Qwen3.5-0.8B-heretic-NVFP4", "messages": [ {"role": "user", "content": "Hello, who are you?"} ] }'
Risks, Limitations & Disclaimer
Safety behavior has been intentionally removed
This model's refusal behaviors have been removed via weight orthogonalization (directional ablation
of the residual stream). It will produce content on topics the base model declines, including content
that may be dangerous, harmful, or illegal. This is the explicit purpose of the modification, not
a defect.
Intended use
This model is intended for research and evaluation purposes: alignment research,
red-teaming, evaluation of safety methods, and study of refusal mechanisms in language models.
Key limitations
- No built-in safety alignment. This model should be treated as having no guardrails.
Apply external safety filtering on both prompts and completions if exposing to others.
- Ablation is not selective. The removed direction also affects appropriate hedging on
medical, legal, and financial questions — the model may assert confidently where the base model
would recommend consulting a professional.
- Refusal suppression is inconsistent. The same request may be answered or refused across runs,
system prompts, or generation parameters. Do not treat this model as either reliably safe or
reliably compliant.
- Blackwell required. NVFP4 requires SM120+ (Blackwell architecture). This model will not
load on Ampere or Hopper GPUs.
- Inherited limitations. All base-model limitations carry over: hallucination,
training-data cutoff, and cultural/linguistic bias.
Responsibility
The end-user assumes full responsibility for all downstream use of this model and any content it
generates. Compliance with applicable laws and regulations in your jurisdiction is the user's
responsibility. Neither the base-model authors (Qwen Team, Alibaba Cloud) nor the ablation tool
authors endorse this derivative.
License & Attribution
Released under Apache-2.0, inherited from
Qwen/Qwen3.5-0.8B.
Citation
@misc{qwen35_08b_heretic_nvfp4, title = {Qwen3.5-0.8B-heretic-NVFP4}, author = {cbrigance}, year = {2026}, note = {NVFP4 quantization of abliterated Qwen/Qwen3.5-0.8B produced with heretic-llm}, url = {https://huggingface.co/cbrigance/Qwen3.5-0.8B-heretic-NVFP4}} @misc{arditi2024refusal, title = {Refusal in Language Models Is Mediated by a Single Direction}, author = {Andy Arditi and Oscar Obeso and Aaquib Syed and Daniel Paleka and Nina Panickssery and Wes Gurnee and Neel Nanda}, year = {2024}, eprint = {2406.11717}, archivePrefix = {arXiv}, primaryClass = {cs.LG}}
Changelog
- 2026-11-15 — Initial model card.