GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
wrldsuksgo2mars
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
wrldsuksgo2mars
Model Tree
deepseek-ai/DeepSeek-V4-Flash-0731
Input Modalities
Output Modalities
Supported Functionality
The quantization used 1,421 prompts totaling approximately 1.08 million tokens. The corpus combined English- and Chinese-focused multilingual Wikipedia material, open-source code review and transformation prompts, and reasoning prompts drawn from mathematics, science, computing, philosophy, economics, law, and politics. Code examples emphasized Python, C++, CUDA, C, and Rust while retaining broader language coverage.
Quantization was performed layer by layer with quantized-prefix replay, so each later block was calibrated against activations produced through the already-quantized prefix. Natural router selections supplied expert activations; sparsely routed experts were supplemented from lower-ranked router candidates up to a 1,024-row target, with an identity-Hessian residual only for any remaining shortfall. Hessians were accumulated as FP32 activation Gram matrices, symmetrized in FP64, and used by the EXL3 trellis quantizer with sigma_reg = 0.025.
No standardized quality benchmark is claimed for this release. As with any low-bit quantization, behavior can differ from the source model and should be evaluated for the intended workload.
The validated vLLM recipe builds a pinned native arm64/SM121 image and serves this standard, unsliced checkpoint with its own full K2 dSpark drafter. At gpu_memory_utilization=0.85, its qualified default accepts a 1,000,000-token request and exposes a measured 1,183,301-token KV pool. Reproducible build instructions, the compact-cache runtime patches, and raw correctness/performance measurements are included in the recipe.
The checkpoint follows the source DeepSeek V4 Flash architecture: 43 blocks, 4,096 hidden size, 256 routed experts with top-6 routing, one shared expert, integrated dSpark speculative blocks, and a configured maximum position length of 1,048,576 tokens.
An inference runtime must support DeepSeek V4 and EXL3 with the MCG codebook. The files use the standard Hugging Face sharded checkpoint layout and include the EXL3 tensor-storage mapping needed to load the quantized expert projections.
Use the tokenizer and prompt formatting described by the original model card. The original model and this quantized checkpoint are distributed under the included MIT license.