What's in the repo
≈96 GB of sharded Safetensors plus config and tokenizer. Everything follows the
original 0731 format, so tooling that understands the base model understands this one.
The config and tokenizer load fine with AutoConfig / AutoTokenizer; per-layer
pruning choices are recorded in SELECTION.json.
DSpark
The model's built-in speculative decoder (three extra MoE stages that draft tokens
ahead) is preserved untouched under its mtp.* tensor namespace. In the GGUF release
it ships as a separate optional file — see the companion repo.
How it was made
One pruning step, straight from the original — no cascading. Expert importance was
measured by running deepseek-ai/DeepSeek-V4-Flash-0731
over a ~5-million-token calibration mix (multi-turn dialogs, thinking and direct modes,
rendered with the model's own chat encoder). The strongest experts of every domain were
protected from pruning, the survivors were carried over byte-identical, and the
router was re-balanced to keep the original selection behavior.
Table with columns: Calibration domain, Share| Calibration domain | Share |
|---|
| Code | 35% |
| Agentic / tool use | 19% |
| Multilingual chat | 16% |
| Math | 8% |
| General chat | 6% |
| Roleplay | 6% |
| Russian | 5% |
| Long docs | 4% |
This line replaces the earlier cascaded REAM builds (now archived under -exp names),
which degraded badly in multi-turn use.
Smoke results
Every scenario is a live multi-turn conversation, run on the companion 2-bit GGUF build — the only runnable form of this checkpoint. Treat the results as a lower bound for this full-precision source (raw evidence ships in the companion repo's SMOKE_REPORT.json).
Table with columns: Scenario, Result| Scenario | Result |
|---|
| Russian wordplay, multi-turn | ✅ |
| English → Russian code-switching | ✅ |
| Code Q&A over a 4k-token file | ✅ |
| Tool calling (DSML) | ❌ |
| Russian multi-turn reasoning | ✅ |
| Spanish creative writing | ✅ |
| Code refactoring | ❌ |
| Chinese summarization | ✅ |
| Long-dialog focus (drift check) |
Limitations
- Runs only through the DS4-fork ecosystem; this repo is the archival/source form.
- Pruning is training-free: rarely-used specialist skills of the removed experts are
gone by design. See the smoke table for what was verified.