GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Nickyang
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Nickyang
Model Tree
deepseek-ai/DeepSeek-V4-Flash-0731
Input Modalities
Output Modalities
Supported Functionality
DeepSeek-V4 does not ship a Jinja chat template. Prompt encoding is handled by
the official reference implementation, which is included in this repository as
encoding/encoding_dsv4.py (copied unmodified from the
base model repository):
python
from encoding.encoding_dsv4 import encode_messages prompt = encode_messages([ {"role": "user", "content": "Explain mixture-of-experts routing."}])See encoding/README.md for multi-turn, tool calling and reasoning formats, and
the base model repository for the reference inference stack. Loading the FP4
weights requires a runtime that supports the base model's quantized layout.
RAZOR asks whether the surviving computation can replace an expert's function, rather than how often or how strongly the expert fires. For a token routed to the set S with normalized weights wj, let be the routed mixture and the consensus residual of expert . Deleting a selected expert makes the router promote its highest-ranked unselected expert , whose pseudo-weight is its score divided by the original selected score sum. With the routed-output scale, the exact local output change is
Scores are aggregated by conditional root mean square over the calibration tokens routed to each expert, and the highest-scoring experts are retained per layer. See the paper for the derivation and the code for the implementation.
Calibration used 32,768-token rows from RazorCal, the 2,048-sample multi-domain corpus released with RAZOR.
bash
pip install razor-moe razor saliency --model <path-to-DeepSeek-V4-Flash-0731> \ --data data/RazorCal.json --max-len 32768 --out out/salrazor prune --model <path-to-DeepSeek-V4-Flash-0731> --saliency out/sal \ --method razor --ratio 0.5 --out out/prunedkept_expert_indices.json is the keep-set manifest this checkpoint was built
from, in the format razor verify expects:
bash
razor verify --pruned . --src <path-to-DeepSeek-V4-Flash-0731>Selection depends on the calibration draw, so an independent run reproduces the procedure rather than this exact expert set.
Expert pruning is lossy. Benchmark retention and predictive fidelity do not guarantee stable generation: the paper reports that responses change in diversity, formatting and termination behaviour even where task accuracy is largely preserved. Evaluate on your own workload before deploying.
This checkpoint was validated by tensor-level checks against the pruning plan. The logprob equivalence gate used for the other backbones was not applicable here, because it renders prompts through a chat template that this model family does not use.
The calibration corpus is multi-domain but finite, so behaviour on domains far from it is not characterised by the released measurements.
This is a derivative of
DeepSeek-V4-Flash-0731,
released under the MIT license, and is distributed under the same license. The
retained weights are the base model's own weights; RAZOR contributes the expert
selection, not new parameters. encoding/ is redistributed unmodified from the
base model repository under its license.
The RAZOR code is Apache-2.0. RazorCal records remain subject to their applicable upstream terms; see data/LICENSE-DATA.
bibtex
@misc{song2026razorpruningreplaceableexperts, title={RAZOR: Pruning Replaceable Experts in LLMs}, author={Mingyang Song and Mao Zheng}, year={2026}, eprint={2609.30465}, archivePrefix={arXiv}, primaryClass={cs.LG}, url={https://arxiv.org/abs/2609.30465},}pip install razor-moe razor saliency --model <path-to-DeepSeek-V4-Flash-0731> \ --data data/RazorCal.json --max-len 32768 --out out/salrazor prune --model <path-to-DeepSeek-V4-Flash-0731> --saliency out/sal \ --method razor --ratio 0.5 --out out/prunedrazor verify --pruned . --src <path-to-DeepSeek-V4-Flash-0731>