GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
taiyi-lab
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
taiyi-lab
Model Tree
deepseek-ai/DeepSeek-V4-Flash
Input Modalities
Output Modalities
Supported Functionality
The uploaded weights were converted from the Megatron-SWIFT MCore checkpoint with megatron export --to_hf true.
Example paths used for this export:
bash
MODEL=/mnt/models/basemodel/DeepSeek-V4-FlashCKPT=/mnt/data/output/megatron_8node_deepseek_v4_flash_v10_tau_style_v4_flashmla_16k_ppbal_mcoreckpt/v0-20260711-123643/checkpoint-5934OUT=/mnt/data/output/megatron_8node_deepseek_v4_flash_v10_tau_style_v4_flashmla_16k_ppbal_mcoreckpt/v0-20260711-123643/checkpoint-5934-hf-safetensors
For this DeepSeek-V4-Flash FP8 checkpoint, the export container used a small runtime patch via PYTHONPATH:
bash
export PYTHONPATH=/mnt/data/patches/deepseek_hf_export:${PYTHONPATH:-}export DEEPSEEK_EXPORT_PATCH_REQUIRED=1
The patch handled two compatibility details:
mcore_bridge.GPTBridge.save_weights compatibility with the current swift export call signature.wo_a weights when PP > 1.The core command was:
bash
megatron export \--to_hf true \--model "$MODEL" \--mcore_model "$CKPT" \--output_dir "$OUT" \--exist_ok true \--model_type deepseek_v4 \--template deepseek_v4 \--agent_template deepseek_v4 \--tensor_model_parallel_size 1 \--pipeline_model_parallel_size 8 \--pipeline_model_parallel_layout 'Et*6|t*6|t*6|t*6|t*6|t*5|t*5|t*3,m,L' \--context_parallel_size 1 \--expert_model_parallel_size 8 \--expert_tensor_parallel_size 1 \--sequence_parallel true \--max_length 16384 \--bf16 true \--fp8_format e4m3 \--fp8_recipe blockwise \--fp8_param_gather true \--mtp_num_layers 1 \--moe_grouped_gemm true \--moe_permute_fusion true \--attention_backend flash \--apply_dsa_kernel_fusion true \--test_convert_precision false
The export was launched on 8 nodes, one container per node, using the same image and shared filesystem mounts:
bash
IMG=modelscope-qwen36:deepseek-v4-flashmla-nvdevWORK=/mnt/cephfs/ubuntu/qwen36sftMODELS=/mnt/cephfs/ubuntu/modelsSCRIPT=/mnt/data/scripts/export_deepseek_v4_flash_5934_hf.sh# inside each node containerbash "$SCRIPT" <node_rank_0_to_7>
The container was started with host networking, host IPC, all GPUs, and the shared data/model mounts:
bash
docker run -d \--gpus all \--network host \--ipc=host \--privileged \--shm-size=64g \--ulimit memlock=-1 \--ulimit stack=67108864 \-v ${WORK}:/mnt/data \-v ${MODELS}:/mnt/models \${IMG} \bash ${SCRIPT} <node_rank>
A successful export writes a HuggingFace model folder like:
text
config.jsongeneration_config.jsontokenizer.json / tokenizer_config.json / special_tokens_map.jsonmodel.safetensors.index.jsonmodel-00001-of-00060.safetensors...model-00060-of-00060.safetensors
You can sanity-check the result with:
bash
python - <<'PY'from transformers import AutoConfig, AutoTokenizerpath = "/path/to/checkpoint-5934-hf-safetensors"print(AutoConfig.from_pretrained(path, trust_remote_code=True).model_type)print(type(AutoTokenizer.from_pretrained(path, trust_remote_code=True)).__name__)PY
The exported safetensors checkpoint was tested with SGLang using DeepEP:
bash
SGLANG_DSV4_FP4_EXPERTS=0 \SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=256 \sglang serve \--trust-remote-code \--model-path /path/to/checkpoint-5934-hf-safetensors \--tp 4 \--dp 4 \--enable-dp-attention \--moe-a2a-backend deepep \--cuda-graph-max-bs-decode 128 \--max-running-requests 256 \--deepep-config '{"normal_dispatch":{"num_sms":96},"normal_combine":{"num_sms":96}}' \--host 0.0.0.0 \--port 30000