Garbled output
The base model garbles in very short conversations. Thinking mode with no system prompt (vLLM's default when a
client sends no system message) and short system prompts both trigger it. It shows up as stray CJK tokens, headings or
newlines in the middle of a sentence, glued words, or a leaked template:
base 1017, greedy: Hello! How can\n can I help you today?base 1017, T=0.6: Hi! How can## Heading\n\ncan I help you today?
This revision repairs it. vLLM 0.32, 4×H200, TP=4, CUDA graphs. A request counts as garbled if it shows token salad,
a stray script or special token, a newline/heading/tag right after a function word, or glued tokens, in the answer
or the reasoning. Fluent text in another language is counted separately and not included here.
Table with columns: Scenario, base 1017, v1, this revision| Scenario | base 1017 | v1 | this revision |
|---|
OpenCode request (long system prompt, 11 tools), user test, OpenCode's default sampling | 0/360 | 25/360 | 0/300 |
Thinking, no system prompt, user test, T=1 | 112/500 | – | 0/500 |
Thinking, no system prompt, test / hi, T=0.6 | 37/200 (eager) | – | 0/300 |
Chat, system You are a helpful assistant., user hello, T=1 | 98/100 | – | 0/150 |
Same, user hi | 99/100 | – | 0/150 |
Same, greedy (hello / hi / hey) | 6/6 | – | 0/6 |
All 28 cells (adds hey/hello, short system + thinking, 12 unseen one-line prompts) | – | – | 4/3,086 |
Same 28 cells, eager mode (--enforce-eager) | – | – | 6/3,086 (one is a false positive: syllable counting) |
The fix is not literally zero. The handful of remaining cases sit at the start of the reasoning after one-word
prompts such as help or ok, for example This is a## Step 1. In our tests on the base model, adding a real system
prompt of about 20 tokens removed the effect almost completely. Per-cell counts are in evals/v4/garble_vllm.json.
What changed in v4
- Ablation, layers 10–38 (v1: 10–39). v1 also edited layer 39. Layer 39 alone writes the model's massive
residual channels (529, 2455, 2953, 737), and v1 flipped those writes. Under vLLM's CUDA-graph decode this sent
agentic sessions into token salad. v4 projects out the same rank-6 chat+thinking refusal subspace, but with
those four channels zeroed and the directions re-orthogonalised. It edits layers 10–38 with λ = 3.5 and leaves
layer 39 exactly as in the base.
<|System|> embedding row repaired (id 128799). In the base checkpoint this row looks untrained: its norm
matches the unused placeholder rows. With almost nothing else in context, the model then collapses into a
"start of document" state at open-class word slots. Thinking mode always opens a System segment for its effort
line, so this affects every short thinking-mode request. The row was replaced by a vector found with a
forward-only CMA-ES search in a 32-dimensional subspace of embedding rows. The search minimised the
attractor probability at those slots while bounding the KL divergence on contexts the model already handled well:
long and OpenCode system prompts, and 20-token system prompts.
- DSpark noise token moved to 128798. The built-in DSpark drafter uses
<|System|>'s id as its noise/mask
input and shares the embedding table. The original 128799 row is copied into the unused placeholder row 128798,
and dspark_noise_token_id is set to 128798 in config.json and inference/config.json. The drafter therefore
sees exactly the base inputs. If your serving recipe hard-codes dspark_noise_token_id: 128799, change it to
128798.
What was edited, and what is checkable
layers.10..38.attn.wo_b.{weight,scale}: 29 FP8 tensors (block 32×32, ue8m0 scales), re-quantized after the
projection. Mean relative Frobenius change 0.122 (max 0.134).
embed.weight rows 128799 and 128798 (2 × 5120 bf16 values, in model-00002-of-00048.safetensors).
config.json, inference/config.json: dspark_noise_token_id 128799 → 128798.
- Everything else is byte-identical to the base. That includes layer 39, the MXFP4 routed experts, the Engram
tables, the vision tower, the hash-MoE gate biases and the DSpark draft blocks. Diff any shard against the official
repository to check.
ABLIT_META.json records the recipe.
Usage
Serve exactly like the official checkpoint. Tested with vLLM 0.32.0 on 4×H200:
vllm serve windowsxp811203/DeepSeek-V4.1-Flash-1017-Abliterated \ --tensor-parallel-size 4 --tokenizer-mode deepseek_v41 \ --reasoning-parser deepseek_v41 --tool-call-parser deepseek_v41 --enable-auto-tool-choice \ --speculative-config '{"method":"dspark","num_speculative_tokens":5}'
- DSpark. Measured on the layer 10–38 ablation with the same drafter inputs: fixed-k with CUDA graphs reached
mean acceptance length 3.16, and the adaptive-verification recipe with full CUDA graphs reached 3.12. Both had 0
garbled outputs. Not re-measured on this revision.
- No Jinja chat template. Use the repo's
encoding/encoding.py (copied from the base) or deepseek-recipe.
Thinking is selected by the prompt format.
- Images work as in the base (DeepSeek-ViT).
Honest notes
- Short-context garbling is reduced, not eliminated. About 0.15% of very short requests still garble, against
roughly 25% to 98% for the base model in the same cells. A real system prompt makes it rarer still.
- The model sometimes answers in Chinese. Like the base, it occasionally replies in Chinese to a one-word
English message, or reasons in Chinese.
- Thinking-mode refusal numbers are weaker evidence. They come from a 96-token reasoning prefix; the chat and
HarmBench results are the headline.
- The evaluation harness is DeepSeek's reference implementation: greedy, slow, not a production engine. The
garbling numbers come from vLLM.
Intended use and limitations
Research on refusal mechanisms, red-teaming, and deployments that bring their own policy layer. Not for unmoderated
end-user exposure. Everything in the base model's limitations applies.
Files
evals/v4/: per-cell garbling counts and the regression summary for this revision. Prompts and model outputs are
not included.
evals/ (top level): v1's per-prompt refusal records, kept for reference.
scripts/: v1's workbench and export scripts.
Credits
DeepSeek-AI for V4.1-Flash-1017 (MIT). lovesenko for the original V4 weight-space ablation recipe. mrfakename, pqhaz
and dealignai for earlier V4.1 ablations whose methods informed the sweep design. Community discussion #33 on the
base repository first documented the short-greeting garbling.
License
MIT, inherited from the base model.