GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
cyijun2k
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
cyijun2k
Model Tree
deepseek-ai/DeepSeek-V4-Flash-0731
Input Modalities
Output Modalities
Supported Functionality
layers.0 through
layers.2). Three is the minimum compact layout that retains the first real
sparse MLA compressor/indexer layer.mtp.0 through mtp.2).hidden_act="silu" and swiglu_limit=10.0.[0, 1, 2], Markov rank 256, block size
5, and three draft stages.[0, 0, 4, 0, 0, 0]; layer 2 includes
the real compressor and Lightning Indexer tensor layouts.The checkpoint contains 14,025 tensors and 24,313,583,928 bytes of tensor data (22.65 GiB).
The generator pins
sakamakismile/DeepSeek-V4-Flash-0731-Abliterated-NVFP4
at revision 4b15c44c31f8dacdf7b0c2fa42e20d03c5f5859b for the DSpark-inclusive
schema. Header-only validation showed:
nvidia/DeepSeek-V4-Flash-NVFP4
revision e3cd60e7de98e9867116860d522499a728de1cf9 for global tensors and target
layers 0–2;tensor_manifest.json records every expected header, and
generate_shape_mock.py reproduces the fixture.
Test host: NVIDIA GB10, compute capability 12.1. The cached
vllm/vllm-openai:nightly image was started with:
text
--moe-backend flashinfer_b12x--linear-backend cutlass--kv-cache-dtype fp8--speculative-config '{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic"}'
The unmodified nightly correctly reproduced its current integration guard:
text
Model sets swiglu_limit=10.0, but the explicitly requestedmoe_backend='flashinfer_b12x' does not apply the SwiGLU clamp.
With a clamp-capable FlashInfer B12X checkout mounted and the stale vLLM capability guard bypassed only inside the disposable test container, vLLM:
FLASHINFER_B12X for the target NVFP4 experts;DSpark draft model loaded: 96 params).The stock nightly did not complete a forward pass on GB10 because its MHC
DeepGEMM path later failed with Unsupported architecture, before reaching the
MoE forward. That limitation is independent of this checkpoint's B12X clamp
schema and is recorded here to avoid overstating end-to-end validation.
Use this checkpoint to reproduce loader selection, tensor remapping, memory planning, and kernel integration behavior. Since the weights are random, any generated tokens are meaningless even when a runtime completes inference.