GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
RedHatAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
RedHatAI
Model Tree
deepseek-ai/DeepSeek-V4-Pro
Input Modalities
Output Modalities
Supported Functionality
This model is intended for deployment with vLLM and requires the following branch: https://github.com/vllm-project/vllm/pull/41276. You can serve the model using
bash
vllm serve RedHatAI/DeepSeek-V4-Pro-NVFP4-FP8-BLOCK --tensor_parallel_size 8 --kv_cache_dtype=fp8
This model was created using LLM Compressor. The example script can be found in examples/quantizing_moe/deepseek_v4_pro_example.py [DSV4] DeepSeekV4 Pro. Quantizing the model with data parallelism and 6xA100 takes about 3 hours.
| Benchmark | deepseek-ai/DeepSeek-V4-Pro-Base | deepseek-ai/DeepSeek-V4-Pro | RedHatAI/DeepSeek-V4-Pro-NVFP4-FP8 |
|---|---|---|---|
| GPQA | 90.1 | 0.93 (380/792 samples) | |
| GSM8K | 91.1 | 92.6 | 91.0 |