DeepSeek-V4.1-Flash-1017-bf16 API & Inference Endpoint

Differences from upstream

quantization_config removed from config.json. An engine that reads it allocates fp8 and MXFP4 parameters, which bfloat16 weights cannot fill.
The two engram tables (layers.1.engram.embed and layers.14.engram.embed, ~98B parameters each) are not dequantized: they keep the upstream fp8 e4m3 weight and 1x32 ue8m0 .scale, because vLLM's engram is fp8-only. In bfloat16 they would add ~197 GB.
All other .scale tensors are dropped (47,587 of them, folded into the bf16 weights).
Everything else is copied unchanged: fp32 tensors (mHC, attention sinks, biases), already-bf16 tensors, the vision tower, the MTP / DSpark weights, the tokenizer, encoding/ and inference/.
Tensor dtypes in the export: 48,107 bf16, 387 fp32, 2 fp8 e4m3 and 2 e8m0 (48,498 tensors in total).

Verification

Tensor names, shapes and dtypes match the index and safetensors headers of PrimeIntellect/DeepSeek-V4.1-Flash-bf16 exactly (same architecture).
160 dequantized tensors (dense fp8 from layers 0 / 20 / 39 and MTP, shared experts, routed experts from layers 0 / 20 / 39 and MTP) were checked against an independent numpy decode (e4m3 and e2m1 bit formulas times 2^(e8m0 - 127) per block); all are bit-exact.
Non-quantized tensors, including both engram tables, are byte-identical to the source.

Use

Same layout as PrimeIntellect's export, so the same usage should apply (vLLM without quantization_config, optionally with online fp8 via quantization = "fp8"; prime-rl model.name). Not yet tested by us on this 1017 export.

Credits and license

The dequantization math is prime-rl's src/prime_rl/trainer/models/deepseek_v4/dequantize.py (commit fffdbe98dc4a34044376c8e09c54406b349d195f), used unmodified. The layout follows PrimeIntellect's DeepSeek-V4.1-Flash-bf16 export.

MIT, inherited from upstream. The upstream LICENSE is included verbatim.

Differences from upstream

quantization_config removed from config.json. An engine that reads it allocates fp8 and MXFP4 parameters, which bfloat16 weights cannot fill.
The two engram tables (layers.1.engram.embed and layers.14.engram.embed, ~98B parameters each) are not dequantized: they keep the upstream fp8 e4m3 weight and 1x32 ue8m0 .scale, because vLLM's engram is fp8-only. In bfloat16 they would add ~197 GB.
All other .scale tensors are dropped (47,587 of them, folded into the bf16 weights).
Everything else is copied unchanged: fp32 tensors (mHC, attention sinks, biases), already-bf16 tensors, the vision tower, the MTP / DSpark weights, the tokenizer, encoding/ and inference/.
Tensor dtypes in the export: 48,107 bf16, 387 fp32, 2 fp8 e4m3 and 2 e8m0 (48,498 tensors in total).

Verification

Tensor names, shapes and dtypes match the index and safetensors headers of PrimeIntellect/DeepSeek-V4.1-Flash-bf16 exactly (same architecture).
160 dequantized tensors (dense fp8 from layers 0 / 20 / 39 and MTP, shared experts, routed experts from layers 0 / 20 / 39 and MTP) were checked against an independent numpy decode (e4m3 and e2m1 bit formulas times 2^(e8m0 - 127) per block); all are bit-exact.
Non-quantized tensors, including both engram tables, are byte-identical to the source.

Use

Credits and license

MIT, inherited from upstream. The upstream LICENSE is included verbatim.

DeepSeek-V4.1-Flash-1017-bf16

README

Differences from upstream

Verification

Use

Credits and license

Explore FriendliAI today

README

Differences from upstream

Verification

Use

Credits and license