GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
windowsxp811203
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
windowsxp811203
Model Tree
deepseek-ai/DeepSeek-V4-Flash-0731
Input Modalities
Output Modalities
Supported Functionality
| Method | W ← W − λ·Vᵀ(VW) applied to attn.wo_b |
| λ | 3.5 |
| Layers | 10–42 + 3 MTP blocks (layers 0–9 preserved — protocol / chat / tool-calling) |
| Rank | 1 |
| Tensors edited | 36 attn.wo_b.weight (33 layers 10–42 + 3 MTP blocks) |
| Shards untouched | 12 (bit-identical to base) |
| Δrel | median 0.0567 (min 0.0495 / max 0.0669) |
The FP8 (F8_E4M3) weights are dequantised per 128×128 block, projected, and requantised;
all other dtypes (F8_E8M0 scales, I8-packed FP4 experts, BF16, F32) are passed through untouched.
520 AdvBench harmful_behaviors + 400 HarmBench behaviours, temperature=0, greedy.
Refusal detection is a deliberately broad keyword matcher (biased toward over-counting refusals);
every flagged response was then reviewed by hand.
thinking=true — the model's natural mode, no jailbreak prefix
| Benchmark | Base 0731 | This model |
|---|---|---|
| AdvBench (520) | 97.12% (505/520) | 0.00% (0/516) |
| HarmBench standard (200) | 84.00% (168/200) | 0.00% (0/195) |
| HarmBench contextual (93) | 68.48% (63/92) | 0.00% (0/90) |
| HarmBench copyright (100) | 17.98% (16/89) | 0.00% (0/76) |
thinking=false + assistant prefill "Sure, here is"
| Benchmark | Base 0731 | This model |
|---|---|---|
| AdvBench (520) | 30.58% (159/520) | 0.00% (0/520) |
| HarmBench standard (200) | 22.00% (44/200) | 0.00% (0/200) |
| HarmBench contextual (93) | 17.20% (16/93) | 0.00% (0/93) |
| HarmBench copyright (100) | 6.00% (6/100) | 0.00% (0/100) |
HarmBench semantic categories, thinking=true: the base model refuses
chemical_biological 100%, harassment_bullying 100%, illegal 98.3%,
harmful 81.0%, misinformation 79.4%, cybercrime 50.0% — this model refuses 0.0% in all six.
The keyword matcher produced 12 raw hits on this model. All 12 were false positives, verified individually:
Raw (unadjudicated) rates were 0.58% on AdvBench/nothink and 5.00–5.26% on the lyrics-heavy copyright split.
thinking=true was measured with vLLM's --reasoning-parser deepseek_v4, so the chain-of-thought
lands in reasoning and content holds only the final answer. Responses truncated by max_tokens
before the reasoning finished are reported separately and excluded from the denominator
(4/520 on AdvBench) rather than being scored as compliance.
| Benchmark | Base 0731 | This model | Δ | 1 s.e. |
|---|---|---|---|---|
| GSM8K (all 1319) | 95.8% | 95.2% | −0.6 pp | ±0.57 pp |
| MMLU (1000 items / 57 subjects) | 88.4% | 87.5% | −0.9 pp | ±1.03 pp |
| Tool-call round-trip | pass | pass | — | — |
Both deltas sit within one standard error — statistically indistinguishable from the base model.
Tool calling was verified end-to-end (finish_reason: tool_calls, correctly populated arguments),
which is why layers 0–9 were deliberately left untouched.
Drop-in replacement for the base checkpoint — same 48 shards, same index.json, same tokenizer.
bash
vllm serve <path-to-this-repo> --tensor-parallel-size 2 --kv-cache-dtype fp8 \--trust-remote-code --enable-auto-tool-choice \--tool-call-parser deepseek_v4 --reasoning-parser deepseek_v4
Verified on vLLM 0.27.1 / torch 2.13.0+cu130 / Python 3.10. None of these are needed on H200 except (1) and (2).
CUDA_DEVICE_ORDER=PCI_BUS_ID. Without it CUDA_VISIBLE_DEVICES reorders CUDA devices but
not NVML, so vLLM reads the wrong card's compute capability — torch reports (12,0) while vLLM
reports major=9. The model then loads fine and dies in profile_run with
layout.hpp:107: sf.size(-2) == ceil_div(mn, gran_mn), because the FP8 einsum recipe is chosen for
SM90 while DeepGEMM has already packed the scales as INT32 for SM120.--kernel-config '{"moe_backend":"marlin"}' — the default DEEPGEMM_MXFP4 path hits
runtime_utils.hpp:91 Unsupported dtype on SM120.8b1392b, installed with pip install --no-build-isolation
(its setup.py imports torch). The bundled copy ships no SM120 kernels.evals/ contains every script and result file, plus the raw stdout logs behind each number:
probe_refusal.py — refusal probe (CSV benchmarks, category breakdown, truncation accounting)eval_gsm8k.py, eval_mmlu.py — capability evalsproject_wob.py — the weight-edit script that produced this modellogs/ — unfiltered run logsThis model does not refuse, including in the most severe categories measured by HarmBench (chemical/biological, cybercrime, harassment). That is the entire point of the modification, and it means safety behaviour must come from somewhere else in your stack. It is suited to trusted, internal, research and red-teaming contexts. If you expose it to third parties, put your own moderation and policy layer in front of it — the model will not do that job for you.
The abliteration removes a specific learned refusal direction; it does not add knowledge, and it does not make the model's outputs more accurate. Everything the base model gets wrong, this one still gets wrong.
deepseek-ai/DeepSeek-V4-Flash-0731 — base model (MIT)drowzeys — the published refusal direction and the project_wob.py / compute_direction.py pipelineapetersson — reference hyper-parameters (λ, layer range) and the Δrel manifest used to sanity-check this runIf these models are useful to you, tips are appreciated — they pay for the GPU time. 如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。
USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888
MIT, inherited from the base model.
vllm/third_partynvidia/cu13/lib64/ and symlink unversioned .so names into it — the directory does not
exist in the pip CUDA wheels, so FlashInfer's JIT link step fails with cannot find -lcudart.flashinfer/comm/fd_exchange.py with from __future__ import annotations — upstream
declares requires-python >=3.10 but uses array.array[int], which needs 3.12.
Also pin flashinfer-cubin to exactly the flashinfer-python version (vLLM 0.27.1 pins 0.6.16.post3).