At a glance
Table with columns: Field, Value| Field | Value |
|---|
| Format | compressed-tensors nvfp4-pack-quantized |
| Quantization | W4A16 ("NVFP4A16"), group size 16, FP8 E4M3 scales, weight only |
| Calibration | none (datafree) |
| Quantized | 96 MLP projections (24 linear-attn blocks) + 32 full-attention q/k/v/o (8 full-attn blocks) = 128 packed NVFP4 tensors |
| Kept in BF16 | linear-attention SSM path (attn_gate, attn_qkv, ssm_alpha, ssm_beta, ssm_out, ssm_conv), all norms, token embeddings, lm_head, vision tower (333 tensors) |
| MTP head | 15 mtp.* tensors carried through in model_mtp.safetensors, bit-verified against the source file |
| Base | xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic (9B hybrid VLM, Apache-2.0) |
Quantized scope in detail
Per linear-attention block (24 blocks): mlp.gate_proj, mlp.up_proj, mlp.down_proj (4 packed groups each: gate/up/down + 1 scale companion pattern per the classic recipe). Per full-attention block (8 blocks at interval 4): self_attn.q_proj, k_proj, v_proj, o_proj. After quantization the packed census is exactly 128 NVFP4 tensors, and every unquantized tensor is byte-identical to the source. The 15 MTP tensors bypass the compressor (known llm-compressor MTP-drop behavior) and are restored from the source's standalone MTP file with bitwise verification.
Checkpoint
- Two safetensors files:
model.safetensors (11,198,705,136 bytes, SHA-256 ca824666fdbd23a01bf7d41659a450f2b03180cfc41e75ce6f98155b5559915b) + model_mtp.safetensors (486,582,848 bytes, SHA-256 e9f83dae0fed4d0633fba652cd7eb942642491a283b11072c3cfc77731e0fae9), under model.safetensors.index.json (1,031 mapped names: 775 trunk + 333 vision + 15 MTP + lm_head + misc — vision and MTP included, nothing dropped).
config.json with quantization_config (quant_method: compressed-tensors, format: nvfp4-pack-quantized), arch Qwen3_5ForConditionalGeneration (Qwen3.5 hybrid text backbone + vision tower).
- Tokenizer, chat template (
chat_template.jinja), preprocessor/processor configs intact.
recipe.yaml: the exact llm-compressor recipe used (datafree, classic patterns + ignores).
Serving
- llama.cpp: use the sibling GGUF repo. A compressed-tensors checkpoint is not directly loadable by llama.cpp; the verified end-to-end path is BF16 → this checkpoint → GGUF (with
convert_hf_to_gguf.py --outtype auto, NVFP4 kept native, GGML type 40).
- vLLM / compressed-tensors engines: the distribution format mirrors published compressed-tensors NVFP4 checkpoints. Load should be auto-detected from the config. We have not exercised a vLLM load ourselves. Our verification was the byte-identity guard, the GGUF path, and full quality/speed benches on the GGUF family; treat a fresh vLLM load as untested until tried.
Verified with these tiers (GGUF form): PPL 4.42–4.53, GSM8K 0.875–0.905 (n=200), MMLU 63.9–68.3% (n=3000), IFEval 30/30 parsed, ToolCap 100% with zero fabricated tools, on two 16 GB Blackwell cards. Details and speed measurements in the GGUF family card.
Provenance
Derivative work, Apache-2.0 throughout; nothing here was trained or fine-tuned. Credit:
- xtools-at for the Heretic decensor tune, and Jackrong for the DeepSeek-V4-Pro-Qwen3.5-9B base.
- Red Hat AI / vLLM project for llm-compressor, used for the NVFP4 quantization.
- esatapedico for the conversion, verification (byte-identity guard, preflight, packed census), and benchmarking.
Follow along & support
I post updates on new conversions, benchmarks, and what I'm working on over on my site. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, you will find the options there. I do this on consumer hardware and like seeing how far it goes. More is on the way.
🌐 esatapedico.com. Updates, work-in-progress, and ways to support.
Note on this card
This model card was written by an AI assistant at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (sizes, configs, commands) against the checkpoint files before relying on it.