Joyfulxp

DeepSeek-V4-Flash-Vision-Exp-Abliterated

Highlights

Table

Base	`deepseek-ai/DeepSeek-V4-Flash-Vision-Exp` (DeepSeek-V4 MoE + 32-layer vision tower)
Architecture	43 layers, hidden 4096, 256 experts (top-6) + 1 shared, hybrid CSA + HCA attention, mHC hyper-connections, inline 3-stage DSpark draft head; vision tower 32×1024 dim, patch 14, downsample 3, `bias_vl`
Context	1,048,576 native (served here at 524,288)
Precision	unchanged — base checkpoint's native precision (FP8 e4m3 block-scaled elsewhere, FP4 routed experts)
Edit	rank-1 orthogonal projection, λ = 2.0, on 43 × `layers..attn.wo_b` + 3 × `mtp..attn.wo_b` (92 tensors)
Untouched	routed + shared experts, `wo_a`, `embed`, `head`, routers, norms, entire 32-layer vision tower + `bias_vl`, all mHC params
Size	this repo: 1.44 GB overlay (+ base ~157 GB required)
Runtime	vLLM (`fraserpricee/vllm:dspark-cu132-vision`), TP=2 across 2× RTX PRO 6000 Blackwell

Validation (abliterated vs original base — measured on this checkpoint)

Refusal & coherence

Table with columns: Metric, Base Vision-Exp, This model (abliterated)
Metric	Base Vision-Exp	This model (abliterated)
Refusal — long-gen mass scan (n=300 AdvBench)	—	0.0 % safety refusals (7 raw detector hits; 6 verified as modality/placeholder misfires, the 1 remaining is an ideological stand on writing an oppressive book — not a safety refusal)
Empty-answer rate (n=300)	—	0.0 %
Generation coherence (40 outputs, chat+think)	clean	0 true garbled spans (10 flags all misreads)
Multi-turn context (8-turn cumulative)	—	8/8 clean, final turn recalls the full arc
(R18 page description)

Long-generation behaviour (8192 thinking budget)

Table with columns: Sample, trigger, explain_qm, write_code, math, Score
Sample	trigger	explain_qm	write_code	math	Score
Base Vision-Exp	✓	✓	✓ (r8.8k)	✓	4/4
This model (λ=2.0)	✓	✓ (r13.7k)	✗ (r30.6k div.)	✓	3/4

The only regression is long write_code generation diverging (~30 k tokens) — a known common consequence of abliteration, present on the published 0731 text sibling as well (92 k divergence). Not a text-coherence loss.

Tool-calling (3-tool schema, DSML native, 6 prompts × 3 reasoning modes = n=18)

Table with columns: Mode, Base Vision-Exp, This model
Mode	Base Vision-Exp	This model
chat	1.000	1.000
think-high	1.000	1.000
think-max	1.000	1.000
Correct tool selected	1.000	1.000

Compliance means: the completion parses via the deepseek_v4 parser, contains a tool call, names a valid offered tool, and supplies every required parameter. Zero difference from base — the edit does not disturb tool calling.

Serving performance (2× RTX PRO 6000 Blackwell, TP=2, single-stream decode, TTFT excluded)

Table with columns: workload, Base Vision-Exp, This model
workload	Base Vision-Exp	This model
technical prose	74.7 tok/s	74.7 tok/s
code	75.6 tok/s	75.5 tok/s
structured JSON	74.9 tok/s	75.1 tok/s
aggregate (8 concurrent)	~262 tok/s (warm)	~255 tok/s (warm)
DSpark draft acceptance	29–38 % (prompt-dependent)	37 %

Serving performance is identical to base — the overlay edits weight values only, never the computation graph. Single-stream decode ~75 tok/s across workloads; ~255–262 tok/s aggregate at 8 concurrent streams. (Earlier cold-start TTFT outliers were prefix-cache warmup artefacts, not model differences.)

How it was built

Refusal directions. Forward hooks on all 43 attn.wo_b modules captured last-token sublayer outputs over AdvBench harmful vs Alpaca-harmless prompts in all three reasoning modes (chat, think-high, think-max). Because wo_b is RowParallelLinear, cross-rank digests were asserted identical on every prompt to guard against capturing a half-direction. Stability: median split-half cosine 0.978–0.990 across layers; held-out AUC 0.974–0.999. Mode agreement: chat↔thinking median cosine 0.46, the two thinking modes 0.90 — matching the measured split on the 0731 sibling, confirming that a single text-mode direction would leave thinking-mode refusal largely intact, hence the three-mode merge.
Merge + bake. The per-mode direction sets were merged and applied as W ← W − λ·r̂(r̂ᵀW) at λ = 2.0 on the FP8 e4m3 weights: dequantize with the block scales, project, then re-quantize holding the original block exponents fixed. Written as a tensor overlay with a repointed model.safetensors.index.json, so the 48 original shards are never rewritten. Bake is deterministic — overlay sha256 58e0461aaf4a387b0026e82e9b39fc73ade52851e1f15288114fbbcf0c0e5350.

The λ operating envelope (heavily validated here)

Table with columns: λ, Result
λ	Result
3.0	Too strong. Long generations garble (quote-storm, semantic break). Reject.
2.5	Equivalent to 2.0 on the 8 k ruler (3/4, same divergent probe) but no added benefit on this model.
2.0	Selected. Full inversion (`
1.9	Under-inversion fuzzy band. Model sits in a half-refuse / half-comply limbo. Reject.
2.05–2.3	e4m3 quant-dead zone — edits swallowed by rounding. Avoid.

So 2.0 is the low edge of a clean operating point: below it is the under-inversion fuzzy band, above it is over-inversion damage. Do not carry over the 0731 text sibling's λ=2.5 — the two are different weight sets and recipes do not transfer.

Serving (vLLM, 2× RTX PRO 6000 Blackwell)

The full model (~157 GB + overlay) needs both 96 GB cards, tensor-parallel = 2:

bash
docker run -d --name ds4-vision-abl --ipc=host --shm-size 32g --gpus all --restart unless-stopped \
  -p 5001:8000 \
  -v /model-dir:/mnt/ubuntulib:ro \
  fraserpricee/vllm:dspark-cu132-vision \
  /opt/venv/bin/python -m vllm.entrypoints.cli.main serve /mnt/ubuntulib/models/DeepSeek-V4-Flash-Vision-Exp-abliterated-l2.0 \
  --served-model-name DeepSeek-Vision-EXP-abliterated --trust-remote-code \
  --tensor-parallel-size 2 --gpu-memory-utilization 0.95 \
  --max-model-len 524288 --max-num-seqs 8 --max-num-batched-tokens 2048 \
  --kv-cache-dtype fp8 --block-size 256 --tokenizer-mode deepseek_v4 \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kernel-config.moe_backend flashinfer_cutlass \
  --speculative-config '{"method":"dspark","num_speculative_tokens":4,"draft_sample_method":"greedy"}' \
  --enable-flashinfer-autotune \
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}' \
  --enable-prefix-caching --enable-chunked-prefill --async-scheduling \
  --reasoning-parser deepseek_v4 --enable-auto-tool-choice --tool-call-parser deepseek_v4 \
  --allowed-local-media-path /data

Blackwell notes (each cost real debugging time):

The serving image must be the vision fork, fraserpricee/vllm:dspark-cu132-vision. The plain SM120 text fork (as used for the 0731 sibling) hard-fails with the vision tower.
--moe-backend flashinfer_cutlass is the working path on this 96 GB Blackwell stack; --attention-backend FLASHINFER_MLA_SPARSE_DSV4 is required for DSV4 sparse MLA.
max-model-len is served at 524,288 here (the vision path adds its own token budget — the tower pads up to vision_max_n_token 384).
--allowed-local-media-path must point at the dir holding images; the vision path reads local file paths, not just base64.
--num_speculative_tokens ≤ dspark_block_size (5): larger values boot but misbehave.
The vision path is much more memory-hungry than text-only; keep gpu-memory-utilization ≤ 0.95.

Findings

The edit is text-path only, yet the visual path also decensors. All 92 edited tensors are attn.wo_b on the text backbone; the 32-layer vision tower and bias_vl are byte-identical. Still, an R18 page is described fully with zero refusal — the text-generation head decodes from a fused carrier whose wo_b was edited, neutralising the last-stage refusal regardless of input modality.
Mode directions diverge here just as on the 0731 sibling. chat↔thinking median cosine 0.46, thinking modes 0.90 — a merged direction is required.
A baked FP8 edit is weaker than a runtime hook (most elements round back). On this model we found the e4m3 dead zone (λ ∈ [2.05, 2.3]) where edits get fully swallowed — a strong reason to tune λ on the baked model, never inherit.

Limitations

Stated plainly.

Capability was gated on refusal / coherence / multi-turn / tool-call only. MMLU-Pro, GSM8K, HumanEval and general benchmarks were not run on this checkpoint. "Refusal, coherence, tool-calling and serving measured" is the claim; "general capability unchanged" is not measured.
Long-context behaviour is unvalidated past multi-turn depth. Our retention test is 8 turns, not an N-shot 168 k needle test. Whether refusal suppression holds at extreme depth is an open question.
Refusal is scored by marker matching on the parsed answer (thinking block excluded) + manual per-row review. It will miss phrasings the marker set does not cover.
The write_code long-generation divergence is the one confirmed capability regression (3/4 vs base 4/4) — a known common consequence of abliteration.
The visual gate is n=1 (one R18 page) — a directional signal, not a rate.
The λ=2.0 operating point is calibrated for the baked FP8 weights of this checkpoint. Do not inherit it for other models, and do not reuse the 0731 sibling's λ=2.5 here.

Files

Table with columns: File, Size, Note
File	Size	Note
`model-overlay-00001-of-00001.safetensors`	1.44 GB	the only differing weight file (92 tensors)
`model.safetensors.index.json`	5.4 MB	repointed — 92 keys → overlay, 72,541 → base shard names
`abliteration_report.json`	27 KB	per-tensor edit audit (sha256, clamped, overflow counts)
`config.json` / `configuration.json` /

The 48 base shards are NOT in this repo. Download them from the base model, then add this repo's overlay + repointed index into the same directory. Removing the overlay + restoring the base index fully reverts the model.

License & credits

MIT (inherited from the base model). Base model: deepseek-ai/DeepSeek-V4-Flash-Vision-Exp. Abliteration follows the residual-direction method (Arditi et al., Refusal in LLMs is mediated by a single direction, NeurIPS 2024); attn.wo_b targeting and the overlay-differential pattern follow and independently replicate cebeuq/DeepSeek-V4-Flash-0731-abliterated — the text-only sibling whose λ=2.5 recipe does not transfer to this vision variant. Serving stack: fraserpricee/vllm:dspark-cu132-vision. Not affiliated with or endorsed by the base-model authors.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Joyfulxp

Model Tree

Base

deepseek-ai/DeepSeek-V4-Flash-Vision-Exp

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

Highlights

Table

Base	`deepseek-ai/DeepSeek-V4-Flash-Vision-Exp` (DeepSeek-V4 MoE + 32-layer vision tower)
Architecture	43 layers, hidden 4096, 256 experts (top-6) + 1 shared, hybrid CSA + HCA attention, mHC hyper-connections, inline 3-stage DSpark draft head; vision tower 32×1024 dim, patch 14, downsample 3, `bias_vl`
Context	1,048,576 native (served here at 524,288)
Precision	unchanged — base checkpoint's native precision (FP8 e4m3 block-scaled elsewhere, FP4 routed experts)
Edit	rank-1 orthogonal projection, λ = 2.0, on 43 × `layers..attn.wo_b` + 3 × `mtp..attn.wo_b` (92 tensors)
Untouched	routed + shared experts, `wo_a`, `embed`, `head`, routers, norms, entire 32-layer vision tower + `bias_vl`, all mHC params
Size	this repo: 1.44 GB overlay (+ base ~157 GB required)
Runtime	vLLM (`fraserpricee/vllm:dspark-cu132-vision`), TP=2 across 2× RTX PRO 6000 Blackwell

Validation (abliterated vs original base — measured on this checkpoint)

Refusal & coherence

Table with columns: Metric, Base Vision-Exp, This model (abliterated)
Metric	Base Vision-Exp	This model (abliterated)
Refusal — long-gen mass scan (n=300 AdvBench)	—	0.0 % safety refusals (7 raw detector hits; 6 verified as modality/placeholder misfires, the 1 remaining is an ideological stand on writing an oppressive book — not a safety refusal)
Empty-answer rate (n=300)	—	0.0 %
Generation coherence (40 outputs, chat+think)	clean	0 true garbled spans (10 flags all misreads)
Multi-turn context (8-turn cumulative)	—	8/8 clean, final turn recalls the full arc
(R18 page description)

Long-generation behaviour (8192 thinking budget)

Table with columns: Sample, trigger, explain_qm, write_code, math, Score
Sample	trigger	explain_qm	write_code	math	Score
Base Vision-Exp	✓	✓	✓ (r8.8k)	✓	4/4
This model (λ=2.0)	✓	✓ (r13.7k)	✗ (r30.6k div.)	✓	3/4

Tool-calling (3-tool schema, DSML native, 6 prompts × 3 reasoning modes = n=18)

Table with columns: Mode, Base Vision-Exp, This model
Mode	Base Vision-Exp	This model
chat	1.000	1.000
think-high	1.000	1.000
think-max	1.000	1.000
Correct tool selected	1.000	1.000

Serving performance (2× RTX PRO 6000 Blackwell, TP=2, single-stream decode, TTFT excluded)

Table with columns: workload, Base Vision-Exp, This model
workload	Base Vision-Exp	This model
technical prose	74.7 tok/s	74.7 tok/s
code	75.6 tok/s	75.5 tok/s
structured JSON	74.9 tok/s	75.1 tok/s
aggregate (8 concurrent)	~262 tok/s (warm)	~255 tok/s (warm)
DSpark draft acceptance	29–38 % (prompt-dependent)	37 %

How it was built

Refusal directions. Forward hooks on all 43 attn.wo_b modules captured last-token sublayer outputs over AdvBench harmful vs Alpaca-harmless prompts in all three reasoning modes (chat, think-high, think-max). Because wo_b is RowParallelLinear, cross-rank digests were asserted identical on every prompt to guard against capturing a half-direction. Stability: median split-half cosine 0.978–0.990 across layers; held-out AUC 0.974–0.999. Mode agreement: chat↔thinking median cosine 0.46, the two thinking modes 0.90 — matching the measured split on the 0731 sibling, confirming that a single text-mode direction would leave thinking-mode refusal largely intact, hence the three-mode merge.
Merge + bake. The per-mode direction sets were merged and applied as W ← W − λ·r̂(r̂ᵀW) at λ = 2.0 on the FP8 e4m3 weights: dequantize with the block scales, project, then re-quantize holding the original block exponents fixed. Written as a tensor overlay with a repointed model.safetensors.index.json, so the 48 original shards are never rewritten. Bake is deterministic — overlay sha256 58e0461aaf4a387b0026e82e9b39fc73ade52851e1f15288114fbbcf0c0e5350.

The λ operating envelope (heavily validated here)

Table with columns: λ, Result
λ	Result
3.0	Too strong. Long generations garble (quote-storm, semantic break). Reject.
2.5	Equivalent to 2.0 on the 8 k ruler (3/4, same divergent probe) but no added benefit on this model.
2.0	Selected. Full inversion (`
1.9	Under-inversion fuzzy band. Model sits in a half-refuse / half-comply limbo. Reject.
2.05–2.3	e4m3 quant-dead zone — edits swallowed by rounding. Avoid.

Serving (vLLM, 2× RTX PRO 6000 Blackwell)

The full model (~157 GB + overlay) needs both 96 GB cards, tensor-parallel = 2:

bash
docker run -d --name ds4-vision-abl --ipc=host --shm-size 32g --gpus all --restart unless-stopped \
  -p 5001:8000 \
  -v /model-dir:/mnt/ubuntulib:ro \
  fraserpricee/vllm:dspark-cu132-vision \
  /opt/venv/bin/python -m vllm.entrypoints.cli.main serve /mnt/ubuntulib/models/DeepSeek-V4-Flash-Vision-Exp-abliterated-l2.0 \
  --served-model-name DeepSeek-Vision-EXP-abliterated --trust-remote-code \
  --tensor-parallel-size 2 --gpu-memory-utilization 0.95 \
  --max-model-len 524288 --max-num-seqs 8 --max-num-batched-tokens 2048 \
  --kv-cache-dtype fp8 --block-size 256 --tokenizer-mode deepseek_v4 \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kernel-config.moe_backend flashinfer_cutlass \
  --speculative-config '{"method":"dspark","num_speculative_tokens":4,"draft_sample_method":"greedy"}' \
  --enable-flashinfer-autotune \
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}' \
  --enable-prefix-caching --enable-chunked-prefill --async-scheduling \
  --reasoning-parser deepseek_v4 --enable-auto-tool-choice --tool-call-parser deepseek_v4 \
  --allowed-local-media-path /data

Blackwell notes (each cost real debugging time):

The serving image must be the vision fork, fraserpricee/vllm:dspark-cu132-vision. The plain SM120 text fork (as used for the 0731 sibling) hard-fails with the vision tower.
--moe-backend flashinfer_cutlass is the working path on this 96 GB Blackwell stack; --attention-backend FLASHINFER_MLA_SPARSE_DSV4 is required for DSV4 sparse MLA.
max-model-len is served at 524,288 here (the vision path adds its own token budget — the tower pads up to vision_max_n_token 384).
--allowed-local-media-path must point at the dir holding images; the vision path reads local file paths, not just base64.
--num_speculative_tokens ≤ dspark_block_size (5): larger values boot but misbehave.
The vision path is much more memory-hungry than text-only; keep gpu-memory-utilization ≤ 0.95.

Findings

The edit is text-path only, yet the visual path also decensors. All 92 edited tensors are attn.wo_b on the text backbone; the 32-layer vision tower and bias_vl are byte-identical. Still, an R18 page is described fully with zero refusal — the text-generation head decodes from a fused carrier whose wo_b was edited, neutralising the last-stage refusal regardless of input modality.
Mode directions diverge here just as on the 0731 sibling. chat↔thinking median cosine 0.46, thinking modes 0.90 — a merged direction is required.
A baked FP8 edit is weaker than a runtime hook (most elements round back). On this model we found the e4m3 dead zone (λ ∈ [2.05, 2.3]) where edits get fully swallowed — a strong reason to tune λ on the baked model, never inherit.

Limitations

Stated plainly.

Capability was gated on refusal / coherence / multi-turn / tool-call only. MMLU-Pro, GSM8K, HumanEval and general benchmarks were not run on this checkpoint. "Refusal, coherence, tool-calling and serving measured" is the claim; "general capability unchanged" is not measured.
Long-context behaviour is unvalidated past multi-turn depth. Our retention test is 8 turns, not an N-shot 168 k needle test. Whether refusal suppression holds at extreme depth is an open question.
Refusal is scored by marker matching on the parsed answer (thinking block excluded) + manual per-row review. It will miss phrasings the marker set does not cover.
The write_code long-generation divergence is the one confirmed capability regression (3/4 vs base 4/4) — a known common consequence of abliteration.
The visual gate is n=1 (one R18 page) — a directional signal, not a rate.
The λ=2.0 operating point is calibrated for the baked FP8 weights of this checkpoint. Do not inherit it for other models, and do not reuse the 0731 sibling's λ=2.5 here.

Files

Table with columns: File, Size, Note
File	Size	Note
`model-overlay-00001-of-00001.safetensors`	1.44 GB	the only differing weight file (92 tensors)
`model.safetensors.index.json`	5.4 MB	repointed — 92 keys → overlay, 72,541 → base shard names
`abliteration_report.json`	27 KB	per-tensor edit audit (sha256, clamped, overflow counts)
`config.json` / `configuration.json` /

DeepSeek-V4-Flash-Vision-Exp-Abliterated

README

Highlights

Validation (abliterated vs original base — measured on this checkpoint)

Refusal & coherence

Long-generation behaviour (8192 thinking budget)

Tool-calling (3-tool schema, DSML native, 6 prompts × 3 reasoning modes = n=18)

Serving performance (2× RTX PRO 6000 Blackwell, TP=2, single-stream decode, TTFT excluded)

How it was built

The λ operating envelope (heavily validated here)

Serving (vLLM, 2× RTX PRO 6000 Blackwell)

Findings

Limitations

Files

License & credits

Explore FriendliAI today

README

Highlights

Validation (abliterated vs original base — measured on this checkpoint)

Refusal & coherence

Long-generation behaviour (8192 thinking budget)

Tool-calling (3-tool schema, DSML native, 6 prompts × 3 reasoning modes = n=18)

Serving performance (2× RTX PRO 6000 Blackwell, TP=2, single-stream decode, TTFT excluded)

How it was built

The λ operating envelope (heavily validated here)

Serving (vLLM, 2× RTX PRO 6000 Blackwell)

Findings

Limitations

Files

License & credits