Evaluation
Table with columns: Benchmark, Full N, Score| Benchmark | Full N | Score |
|---|
| MMLU-Pro | 12032 | 0.6750 |
| GSM8K | 1319 | 0.9257 |
| HumanEval (pass@1) | 164 | 0.8354 |
| MBPP (pass@1) | 500 | 0.5160 |
Multi-turn & higher-context evaluation
Table with columns: Benchmark, Score| Benchmark | Score |
|---|
| Multi-turn (20 curated 3-turn convos / 60 turns, Gemini judge 1-10) | 9.98 / 10 |
| Needle-in-haystack @ 2k / 4k / 8k / 16k / 32k tokens | 100% / 100% / 100% / 100% / 100% |
SWE-bench Lite (oracle-file-context, single-shot, n=30)
Table with columns: Metric, Value| Metric | Value |
|---|
| Resolved | 4 / 30 (13.3%) |
| Completed | 19 / 30 |
| Patch-apply errors | 11 / 30 |
Speculative decoding throughput
Single-stream: ~238 tok/s at temperature=0 (TP=2, DSpark native 5-token speculation, lucifer-cutlass backend).
Safety compliance by category (1000-prompt eval set)
Table with columns: Category, Compliance, Retention| Category | Compliance | Retention |
|---|
| PII (doxing private individuals) | 42 / 78 | 53.8% |
| Self-Harm (suicide methods) | 59 / 70 | 84.3% |
| Radicalization | 62 / 67 | 92.5% |
| Cybercrime | 63 / 67 | 94.0% |
| Hate Speech | 74 / 78 | 94.9% |
| Illegal Drugs | 66 / 69 |
Safety guardrails are highest in PII doxing (46.2% retention) and self-harm (84.3%), where the model defaults to empathetic hotline-style responses and refuses to surface private contact data. In all 12 remaining categories compliance exceeds 92%.
Files
All 48 safetensors shards are included, plus model.safetensors.index.json, config.json, generation_config.json, tokenizer.json, tokenizer_config.json, LICENSE, and the encoding/ and inference/ folders. It loads directly with vLLM / the DeepSeek-V4 inference path.
Usage
This checkpoint is a drop-in replacement for the original weights — it has the same architecture, format, chat-template/encoding, and inference path as deepseek-ai/DeepSeek-V4-Flash-DSpark. Load and serve it with vLLM or the DeepSeek-V4 inference stack.