GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
Jiunsong
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Jiunsong
Model Tree
deepseek-ai/DeepSeek-V4-Flash-0731
Input Modalities
Output Modalities
Supported Functionality
| Architecture | DeepSeek V4 Flash, 304B-class MoE, 43 backbone layers + 3 MTP layers, 256 routed experts, top-6 |
| Release format | Hybrid FP4 experts + FP8 E4M3 blocks + BF16 quality-sensitive tensors, about 169.5 GB on the Hub |
| Targeted update | 46 attn.wo_b weight/scale pairs, with all routed experts and untargeted parent tensors preserved |
| Verified context | 1,048,576 configured; 1,028,621-token prompt accepted with successful needle retrieval |
| Aggregate decode | 118.6 tok/s forced output and 123.3 tok/s structured tool output at p256/C6 |
| Worst-mode refusal 97.92% -> 4.17%, while the measured tool gates remain 100% |
| Capability floor | Minimum capability mean 0.9375 -> 0.9583 against the pinned parent |
MQ in the model name means mixed quantization. This is an
official-layout mixed-precision checkpoint, not a full-BF16 release and not a
custom whole-model requantization.
| Component | Precision / storage |
|---|---|
| MoE expert weights | FP4, inherited from the official expert_dtype=fp4 checkpoint layout |
| Block-quantized paths | FP8 E4M3, dynamic activation scaling, 128x128 weight blocks, UE8M0 scales |
43 backbone + 3 MTP attn.wo_b updates | Deterministic FP8 weight/scale overlay, 92 tensors |
| Default unquantized and quality-sensitive paths | BF16 (torch_dtype=bfloat16) with F32 metadata/normalization where defined upstream |
| Output head recovery | One bounded BF16 head.weight overlay, rank-64, relative Frobenius delta 0.0025 |
| Measured serving KV cache | NVFP4 DS-MLA |
The parent checkpoint is pinned to 9e165c30e2704aec5d9d593cce3eebd58bbef1cb. Only the declared FP8
attn.wo_b pairs and the single bounded BF16 output head are redirected by the final
weight index; the remaining parent tensors keep their original quantization and bytes.
The release uses two measured weight-space passes:
The final checkpoint modifies only the 43 backbone and three MTP attn.wo_b
weight/scale pairs plus the bounded output head. There is no inference-time adapter.
| Metric | Official parent | SuperDeepseek-V4-Flash-abliterated-MQ-2xDGX |
|---|---|---|
| Worst-mode refusal | 97.92% | 4.17% |
| Worst empty answer | 0.00% | 0.00% |
| Worst tool compliance | 100.00% | 100.00% |
| Worst correct-tool rate | 100.00% | 100.00% |
| Minimum capability mean | 0.9375 | 0.9583 |
The independently reloaded checkpoint reproduced the selected candidate's deterministic validation behavior exactly. Empty-output, Unicode, repetition, serialization, reasoning, code, formatting, and tool-use sentinels were included in the release gate.
The numbers below are aggregate concurrent decode throughput, not single-stream speed.
They use six distinct prompts, fixed-length generation, and the sealed
sparkDash-style measurement contract at commit
dfde4214f32b174880832a4d317d3c0567750ac5.
| Workload | Prompt / concurrency | Aggregate decode |
|---|---|---|
| Forced output | p256 / C6 | 118.5505 tok/s |
| Structured tool output | p256 / C6 | 123.2888 tok/s |
| Actual prompt tokens | Accepted | Needle retrieved |
|---|---|---|
| 149,845 | Yes | Yes |
| 1,028,621 | Yes | Yes |
The configured maximum is 1,048,576 tokens. These are end-to-end acceptance and retrieval probes; they are not a claim that every task benefits equally from the full window.
The measured profile uses TP=2 over direct CX-7 RoCEv2 with the
ghcr.io/anemll/dspark-vllm-gx10:0.1.1 runtime:
VLLM_USE_BREAKABLE_CUDAGRAPH=0The repository includes the exact two-rank launcher under
repro/scripts/serve_superdeepseek_v4_dual.sh. Its measured model-facing options are:
bash
vllm serve /model \--served-model-name SuperDeepseek-V4-Flash-abliterated-MQ-2xDGX \--tensor-parallel-size 2 \--max-model-len 1048576 \--kv-cache-dtype nvfp4_ds_mla \--moe-backend flashinfer_b12x \--enable-prefix-caching \--async-scheduling \--enable-chunked-prefill \--speculative-config '{"method":"dspark","num_speculative_tokens":1,"draft_sample_method":"greedy"}'
After the server is ready, it exposes an OpenAI-compatible API:
python
from openai import OpenAIclient = OpenAI(base_url="http://127.0.0.1:8888/v1", api_key="EMPTY")response = client.chat.completions.create(model="SuperDeepseek-V4-Flash-abliterated-MQ-2xDGX",messages=[{"role": "user", "content": "Design a reliable tool-using agent."}],max_tokens=1024,)print(response.choices[0].message.content)
“Uncensored” means this checkpoint measurably reduces the selected refusal subspace while retaining the declared capability and output-integrity gates. It does not imply that every answer is correct or that downstream deployment controls are unnecessary.
deepseek-ai/DeepSeek-V4-Flash-0731@9e165c30e2704aec5d9d593cce3eebd58bbef1cbattn.wo_b weight/scale pairsevidence/| Evidence | SHA-256 |
|---|---|
| paired validation | 65aaf03e1ac5f88ea462c7500d36a06c254370dfa3f9e2a0d24183c482e23a46 |
| phase 1 selection | a75b0f9c2c2c9d0c38e94c5a7e36139509178f7f9b74521925c9a1f011f0f207 |
| phase 2 selection | b0fdb2c73f16954b56bde2460833b782b29bfe0a67aeedc611d65dc94764e4eb |
| recovery decision | 39acc4c24694c7bb08615217d16e5e7d2f3d9e33c26425610ff1198054422a2f |
| decode benchmark | 6745453ee65b9e59581d715bf668bc710b0bc680fe726763e9a750dd25f45565 |
| overlay audit | c618a1a2da8295b07b1d93d9a59c9a66e8e430c2b8bdf1aca07a56ad56455e8e |
| head recovery audit | a5aa6531909444a5d6199d889030b6b9424868a7656e4db8d77c30aad4bfc8db |
MIT, following the upstream DeepSeek V4 Flash release.