GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ProCreations
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
ProCreations
Model Tree
XiaomiMiMo/MiMo-V2.6-Flash-RL
Input Modalities
Output Modalities
Supported Functionality
W4A16_NVFP4 for the experts. No
FP4 activation quantization is requested by this checkpoint configuration.The root tensor payload is 193,964,338,432 bytes, excluding auxiliary files. The larger dense tensors are deliberate: preservation takes priority over reducing every tensor to a low-precision format.
verification.json records an independent, complete audit
of all 65 root shards:
| Check | Verified |
|---|---|
| Expert weight values, identical codes and reconstructed scales | 302,795,194,368 |
| Dense FP8 values, independently decoded and reordered into FP32 | 3,859,808,256 |
| Other tensors, identical bytes | 763 |
Source shard SHA-256 values were also compared with the pinned Hugging Face LFS
manifest. auxiliary-verification.json covers
the retained auxiliary files. Reproduction and independent verification scripts
are in tools/.
Exact stored weights do not imply bit-identical inference. Runtime dtype, activation quantization, attention kernels, accumulation, and sampling can change outputs. In particular, loading FP32 reconstructions with BF16 compute rounds them to the runtime dtype. Native Blackwell FP4 activation kernels must be evaluated separately from this weight conversion. No 60-token/s CPU-offload result is claimed by the weight audit.
The original MXFP4 GGUF and converted NVFP4 GGUF were compared using the same
pinned llama.cpp revision, cuBLAS F32 storage/accumulation, and BF16 KV. The
runtime's cuBLAS handle enables TF32 math for eligible large multiplications;
GGML_CUDA_CUBLAS_COMPUTE_TYPE=f32 alone does not disable that setting.
Both original and converted models used the same math mode. The historical
strict report label means FP4/Q8 activation kernels were bypassed, not that
every multiplication used full-mantissa FP32. See NVIDIA's
TF32 controls.
All original dense values remain FP32. The evaluation covers eight 4,096-token
chunks per domain, scoring the last 2,047 positions in each chunk: 65,504
scored token positions across code, math, English and Chinese.
| Domain | Original PPL | NVFP4 PPL | Top-token agreement |
|---|---|---|---|
| Code | 2.3774 | 2.377430 | 100.000% |
| Math | 1.5172 | 1.517172 | 100.000% |
| English | 9.0561 | 9.056080 | 99.994% |
| Chinese | 12.6352 | 12.635223 | 99.988% |
Perplexity matches within the precision of the original PPL logs. Mean
approximate KL is zero to six printed decimals in every domain. The reference
log probabilities are encoded by llama.cpp as uint16 values with a
max_logit - 16 floor; ties from this encoding can affect the top-token
diagnostic. PPL ratios use the original unclipped PPL logs, not the clipped
log-probability file. These measurements do not establish a general task
accuracy percentage or certify other inference kernels.
See reports/heldout-nvfp4-strict.json,
the associated logs, and
reports/gguf-weight-verification.json.
The latter independently verifies all 302,795,194,368 expert values and all
6,971,406,720 dense values in the two GGUF exports. This canonical release
contains safetensors and an independently verified GGUF package.
Workstation runtime and performance tests are reported separately.
The checkpoint loads with vLLM 0.29.0, tensor parallelism 2, on two RTX PRO 6000 Server Edition 96 GB GPUs. A basic arithmetic completion and executable coding checks have run. This is a runtime smoke test, not a broad quality benchmark. CPU offloading and speculative decoding require separate validation.
bash
vllm serve ProCreations/MiMo-V2.6-Flash-RL-NVFP4 \ --served-model-name mimo-v26-nvfp4 \ --tensor-parallel-size 2 --dtype bfloat16 \ --max-model-len 12288 --max-num-seqs 2 \ --max-num-batched-tokens 512 --gpu-memory-utilization 0.94 \ --enforce-eager --generation-config vllm \ --reasoning-parser mimo --tool-call-parser mimo --enable-auto-tool-choice \ --host 127.0.0.1 --port 30080Use the upstream recommended sampling settings explicitly: temperature 1.0,
top_p 0.95. The original generation_config.json is retained for provenance
and includes do_sample: false; API requests should explicitly enable the
intended sampling settings. Long reasoning can exhaust response limits, so
count truncated responses as incomplete when evaluating real work.
The canonical safetensors release retains the original multimodal and DFlash weights. Inclusion does not certify every runtime's vision/audio generation or DFlash implementation. See the accompanying reports and recipes for the exact tested configurations and outstanding limitations.
The GGUF runtime recipe provides one CPU-offloaded profile for a 96 GB RTX PRO 6000 with 128 GB-class host RAM: 524,288 total context tokens, 131,072 maximum output tokens, scaled E4M3 FP8 target KV, DFlash, and active-prefix reuse. See measured speeds, completed work and limitations. Near-512K retrieval failed and sustained 40–60 tok/s across workloads was not achieved. The weight conversion remains exact; FP8 KV and runtime arithmetic have separate numerical checks.
vllm serve ProCreations/MiMo-V2.6-Flash-RL-NVFP4 \ --served-model-name mimo-v26-nvfp4 \ --tensor-parallel-size 2 --dtype bfloat16 \ --max-model-len 12288 --max-num-seqs 2 \ --max-num-batched-tokens 512 --gpu-memory-utilization 0.94 \ --enforce-eager --generation-config vllm \ --reasoning-parser mimo --tool-call-parser mimo --enable-auto-tool-choice \ --host 127.0.0.1 --port 30080