kirillbilchenko

Nex-N2.5-mini-MXFP4-MxWave

README

License: apache-2.0

Model summary

Table with columns: Property, Value
Property	Value
Base model	`nex-agi/Nex-N2.5-mini`
Base revision	`87420286149d9cce9bd46cd335ef9bda33c37c1b`
Quantized scope	Routed experts in all 40 language layers
Logical expert matrices	30,720
MXFP4 packed/scale tensors	61,440
Source-precision passthrough tensors	946
Output shards	87
Indexed tensor data	22,902,614,752 bytes (21.33 GiB)
Runtime weight path	MXFP4 W4A16 with vLLM `MarlinExperts`
Tested runtime	Official vLLM 0.29.0 on DGX Spark GB10 / SM121

The size above is safetensors tensor payload, not the complete Hugging Face repository or download size.

What was quantized

Table with columns: Component, Stored/executed precision, Treatment
Component	Stored/executed precision	Treatment
Routed-expert `gate_proj`, `up_proj`, `down_proj`	MXFP4 weights, BF16 activations	Quantized
Routers and shared experts	Source BF16	Exact passthrough
Attention and Gated DeltaNet	Source BF16	Exact passthrough
Embeddings, norms, and output head	Source BF16	Exact passthrough
Vision stack

The source config declares MTP support, but the pinned source safetensors index does not contain a separate MTP tensor payload. This card therefore makes no claim that separate MTP weights were preserved or evaluated.

Quantization method

This candidate uses unweighted weight-reconstruction MSE scale search. For each MXFP4 block, MxWave tests scale exponents from floor(log2(p99.5 / 6)) - 4 through floor(log2(p99.5 / 6)) + 1, plus the smallest exponent that avoids clipping, then chooses the candidate with the lowest weight MSE.

It does not use activation, route, LayerNorm-gamma, or Hessian calibration. Calling this particular checkpoint activation-aware or calibration-aware would be inaccurate.

Conversion is bounded by logical expert rows and output-shard payload rather than loading the full source model. The evaluated build used 2,048-row chunks, a 1 GiB materialized host-payload cap, and completed in 517.7 seconds. The cap does not include serializer overhead, allocator state, memory mappings, or the operating-system page cache.

The generic MxWave core is adapter-driven and contains no Nex/Qwen tensor naming. Architecture adapters provide fused-bank names, logical slices, validation, and runtime selectors. Only the Qwen3.5-MoE adapter has been validated so far; testing this model does not establish support or equal quality on another MoE family.

Quick start with vLLM

Download the checkpoint:

bash

HF_XET_HIGH_PERFORMANCE=1 hf download \  kirillbilchenko/Nex-N2.5-mini-MXFP4-MxWave \  --local-dir ./Nex-N2.5-mini-MXFP4-MxWave

The validated environment used the official vLLM 0.29.0 image at digest sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1. The following command reproduces the tested runtime settings; adjust memory and context limits for your workload:

bash

docker run --rm --gpus all --ipc=host -p 8000:8000 \  -v "$PWD/Nex-N2.5-mini-MXFP4-MxWave:/model:ro" \  vllm/vllm-openai@sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1 \  /model \  --served-model-name nex-n2.5-mini-mxwave \  --load-format safetensors \  --dtype bfloat16 \  --linear-backend marlin \  --moe-backend marlin \  --max-model-len 32768 \  --max-num-seqs 4 \  --max-num-batched-tokens 8192 \  --gpu-memory-utilization 0.6 \  --kv-cache-dtype bfloat16 \  --no-enable-prefix-caching

Stock vLLM selected MarlinExperts. A deterministic text smoke test returned the exact requested marker, and a 64x64 red-image smoke test answered Red. These are compatibility checks, not broad text or vision evaluations.

Minimal no-thinking request:

bash

curl http://localhost:8000/v1/chat/completions \  -H 'Content-Type: application/json' \  -d '{    "model": "nex-n2.5-mini-mxwave",    "messages": [{"role": "user", "content": "Explain MXFP4 briefly."}],    "reasoning_effort": "none",    "temperature": 0,    "max_tokens": 256  }'

Intended use and safety

This release is intended for local inference research, runtime qualification, and controlled comparison of expert-only MXFP4 with other representations. It is not independently safety-tuned, and quantization does not remove the base model's risks or restrictions. Review the base model card before deployment, evaluate on your own task distribution, and retain application-level safeguards for high-impact uses.

Evaluation

All comparisons use pinned model revisions and paired inputs. Lower perplexity and lower KL are better. Uncertainty is reported where available.

Paired WikiText-2 prompt perplexity

The frozen protocol used the same 64 WikiText-2 chunks and 61,408 scored tokens for every model.

Table with columns: Model, Prompt PPL, Difference from MxWave
Model	Prompt PPL	Difference from MxWave
BF16 source	8.873219	MxWave is +0.0868%
MxWave expert MXFP4 MSE	8.880923	—
MxWave expert MXFP4 RTN	8.910400	MxWave is -0.3308%
Public NVFP4 control	9.009808	MxWave is -1.4305%

MxWave beat the NVFP4 control on 45 of 64 paired chunks. The paired 95% bootstrap interval for the relative MxWave-minus-NVFP4 PPL difference was [-1.9552%, -0.9302%]. Against BF16, the candidate's 0.0868% point gap was statistically unresolved.

Perplexity here measures observed-token likelihood under this exact windowing protocol. It is not a complete instruction-following or multimodal score.

Frozen no-thinking task screen

The diagnostic suite contained 82 cases, of which 69 had strict automatic scores. All four models saw identical prompts and decoding settings.

Table with columns: Model, Strict score
Model	Strict score
Public NVFP4 control	47/69 (68.12%)
BF16 source	46/69 (66.67%)
MxWave expert MXFP4 MSE	45/69 (65.22%)
MxWave expert MXFP4 RTN	45/69 (65.22%)

MxWave versus NVFP4 had 43 shared passes, two MxWave-only passes, four NVFP4-only passes, and 20 shared failures. The difference was -2.90 percentage points, with exact McNemar p=0.6875 and a stratified paired 95% interval of [-10.14, +4.35] points. This small screen is inconclusive and does not support a general task-quality advantage.

MxWave category results were 6/20 arithmetic, 5/15 logic, 15/15 structured output, 10/10 short retrieval, and 9/9 8K retrieval. There were no request errors.

Exact terminal next-token divergence from BF16

This screen retained the complete 248,320-token next-token distribution at one terminal position for each of 64 frozen contexts.

Table with columns: Metric, MxWave MSE, MxWave RTN, Public NVFP4
Metric	MxWave MSE	MxWave RTN	Public NVFP4
Mean forward KL	0.034157	0.031049	0.040895
Median forward KL	0.022420	0.015636	0.018090
P95 forward KL	0.093739	0.080812	0.122376
Mean reverse KL	0.034255	0.031715

The MSE candidate failed its preregistered exact-KL gate against RTN: mean forward KL was 10.01% higher, and both fixed 32-context halves were worse. The paired interval crossed zero, so the magnitude was unresolved, but this is not a successful MSE result on the primary distribution metric.

Against public NVFP4, MxWave's mean forward KL was 16.48% lower at the point estimate, but the paired relative 95% interval was [-30.90%, +6.66%] and the two fixed halves disagreed (+1.33% and -27.22%). NVFP4 also had better median KL and top-1 agreement. Therefore the data do not establish a KL winner.

Repeated serving measurements

The serving comparison used three paired seeds, disabled prefix caching, BF16 KV cache, a 32K context limit, at most four sequences, and at most 8,192 batched tokens. Each model completed 132 requests with no failures, OOMs, or restarts.

Table with columns: Workload / metric, MxWave MXFP4, Public NVFP4, MxWave delta
Workload / metric	MxWave MXFP4	Public NVFP4	MxWave delta
Model-load memory	21.47 GiB	22.41 GiB	-0.94 GiB
8K prefill, C1 input throughput	6,596.38 tok/s	7,152.96 tok/s	-7.78%
8K prefill, C1 TTFT	1,240.51 ms	1,144.66 ms	+8.47%
32-to-512 decode, C1 output throughput	37.395 tok/s	36.440 tok/s

This is not a pure kernel comparison. MxWave ran weight-only MXFP4 W4A16 using Marlin, while the public NVFP4 control ran W4A4 using FlashInfer/CUTLASS. Throughput conclusions are workload-specific.

Size comparison

Table with columns: Checkpoint, Indexed tensor data, Difference
Checkpoint	Indexed tensor data	Difference
MxWave expert MXFP4 MSE	22,902,614,752 bytes (21.33 GiB)	—
Public NVFP4 control	23,909,493,472 bytes (22.27 GiB)	MxWave is 4.21% smaller

The candidate does not clear the originally proposed 5% size-improvement bar. Its value is the combination of a smaller tensor payload, favorable paired PPL, stock-vLLM compatibility, and different workload trade-offs—not a large compression breakthrough.

Reproduction

The current model-independent CLI was merged in MxWave PR #8. From MxWave commit 4309db0f639eefa3c3c8f126dba2bb89e15dd49e or later:

bash

mxwave-quantize-experts \  --model-dir /path/to/Nex-N2.5-mini-bf16 \  --output-dir /path/to/Nex-N2.5-mini-mxwave-mse \  --device cuda \  --method mse \  --scale-percentile 99.5 \  --mse-clip-depth 4 \  --tensor-row-chunk-size 2048 \  --host-tensor-cap-mib 1024 \  --source-repository nex-agi/Nex-N2.5-mini \  --source-revision 87420286149d9cce9bd46cd335ef9bda33c37c1b

The evaluated weight shards were produced immediately before the CLI was renamed and the manifest/config contract was merged. Its equivalent command was named mxwave-quantize-expert-rtn, despite accepting --method mse. The published release retains those exact evaluated weight shards. Its config adds the now-required group-level string format: "mxfp4-pack-quantized"; this is a metadata normalization and does not alter any weight tensor.

Evaluated-artifact hashes

These hashes bind the artifact used for the numerical results above. The published config.json has a different hash solely because of the group-format normalization described above:

Table with columns: Artifact, SHA-256
Artifact	SHA-256
Ordered checkpoint content	`1d134e5d5c704cc76fe85268ee1f65ece0107e932e9de856c141bdb5fa9e8738`
`config.json`	`8ea02e392011595c2394a26862e51966608abab24fa2b750ce798b4ce67f6351`
Safetensors index	`69d4da6c13645e868454a7e22537e3f2ceef8883cf63817d67e2239145ad293c`
MxWave manifest	`29bd6b121e705e041f1e1520ce12d7e49b4fb4f07c66174907f6b63bd49a76c3`

Published normalized config.json SHA-256: af12954beada319a4527195d003f4b14da9bcfc5ef273a00987afcefe8472963. Published normalized mxwave-manifest.json SHA-256: c4bae2e5d702fee58037d5201fb205b2f71611ca42dbc3c15b26d52ca96047b0. Merged-MxWave release checkpoint fingerprint (all 87 shards hashed): c8ddd02b6015a72fdfd3dd7d0be11dd4393e065141ff335087eb0df196a9f7eb.

Resume validation accepted all 87 completed shards and requantized zero shards.

Evaluation provenance

The compact machine-readable report and file manifest are included in evaluation/results.json and evaluation/SHA256SUMS.

Public control: primitive-ai/Nex-N2.5-mini-NVFP4 at revision ceaa4ce6d942c3d5456322d183b535a0c89a5fa6.
WikiText-2 dataset revision: b08601e04326c79dfdd32d625aee71d232d685c3.
PPL corpus SHA-256: 696cca6b65a171b0a358a4be6732cdfdf2dd6164a32e20fd70e3c13fc4dfae83.
PPL protocol SHA-256: ecc777cb86672560e95196d90e8bc63e2d5c2f7e9eaa9e8b90ba1babefdf91ed.
Exact-KL context manifest SHA-256: 9b72a2aa2b32c13dce3ff2e387d4f4a8d1383a240baf620a58d351566d8c9d3f.
Exact-KL protocol SHA-256: e37d5803afa723cd7505baf407476f36e86c3168e9fbaa69251289e672013f38.
Exact-KL combined comparison SHA-256: .

Limitations

This is expert-only quantization, so the remaining BF16 components limit the total size reduction.
The candidate is near BF16 on the measured PPL protocol, but it is not proven equivalent to BF16 across tasks or distributions.
The small frozen task screen and 64-position exact-KL screen are diagnostic, not broad capability benchmarks.
MSE scale search improved paired PPL over RTN but worsened the primary exact BF16-forward KL point estimate. The evidence is genuinely mixed.
Vision received only a smoke test. Image understanding and generation quality were not benchmarked.
Runtime results compare different activation-precision and kernel paths.
The checkpoint is validated for official vLLM 0.29 on Blackwell. It is not validated for Transformers inference, llama.cpp, Ollama, MLX, or LM Studio.
Only the Qwen3.5-MoE adapter and this checkpoint layout were validated. The engine is adapter-driven, but other architectures require their own adapter and independent evaluation.

License and attribution

This quantized derivative inherits the base model's Apache-2.0 license. Review the base model card for its intended use, restrictions, training information, and original citations.

Quantization and evaluation tooling: MxWave.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

kirillbilchenko

Model Tree

Base

nex-agi/Nex-N2.5-mini

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model summary

Table with columns: Property, Value
Property	Value
Base model	`nex-agi/Nex-N2.5-mini`
Base revision	`87420286149d9cce9bd46cd335ef9bda33c37c1b`
Quantized scope	Routed experts in all 40 language layers
Logical expert matrices	30,720
MXFP4 packed/scale tensors	61,440
Source-precision passthrough tensors	946
Output shards	87
Indexed tensor data	22,902,614,752 bytes (21.33 GiB)
Runtime weight path	MXFP4 W4A16 with vLLM `MarlinExperts`
Tested runtime	Official vLLM 0.29.0 on DGX Spark GB10 / SM121

The size above is safetensors tensor payload, not the complete Hugging Face repository or download size.

What was quantized

Table with columns: Component, Stored/executed precision, Treatment
Component	Stored/executed precision	Treatment
Routed-expert `gate_proj`, `up_proj`, `down_proj`	MXFP4 weights, BF16 activations	Quantized
Routers and shared experts	Source BF16	Exact passthrough
Attention and Gated DeltaNet	Source BF16	Exact passthrough
Embeddings, norms, and output head	Source BF16	Exact passthrough
Vision stack

Quantization method

It does not use activation, route, LayerNorm-gamma, or Hessian calibration. Calling this particular checkpoint activation-aware or calibration-aware would be inaccurate.

Quick start with vLLM

Download the checkpoint:

bash

HF_XET_HIGH_PERFORMANCE=1 hf download \  kirillbilchenko/Nex-N2.5-mini-MXFP4-MxWave \  --local-dir ./Nex-N2.5-mini-MXFP4-MxWave

bash

docker run --rm --gpus all --ipc=host -p 8000:8000 \  -v "$PWD/Nex-N2.5-mini-MXFP4-MxWave:/model:ro" \  vllm/vllm-openai@sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1 \  /model \  --served-model-name nex-n2.5-mini-mxwave \  --load-format safetensors \  --dtype bfloat16 \  --linear-backend marlin \  --moe-backend marlin \  --max-model-len 32768 \  --max-num-seqs 4 \  --max-num-batched-tokens 8192 \  --gpu-memory-utilization 0.6 \  --kv-cache-dtype bfloat16 \  --no-enable-prefix-caching

Minimal no-thinking request:

bash

curl http://localhost:8000/v1/chat/completions \  -H 'Content-Type: application/json' \  -d '{    "model": "nex-n2.5-mini-mxwave",    "messages": [{"role": "user", "content": "Explain MXFP4 briefly."}],    "reasoning_effort": "none",    "temperature": 0,    "max_tokens": 256  }'

Intended use and safety

Evaluation

All comparisons use pinned model revisions and paired inputs. Lower perplexity and lower KL are better. Uncertainty is reported where available.

Paired WikiText-2 prompt perplexity

The frozen protocol used the same 64 WikiText-2 chunks and 61,408 scored tokens for every model.

Table with columns: Model, Prompt PPL, Difference from MxWave
Model	Prompt PPL	Difference from MxWave
BF16 source	8.873219	MxWave is +0.0868%
MxWave expert MXFP4 MSE	8.880923	—
MxWave expert MXFP4 RTN	8.910400	MxWave is -0.3308%
Public NVFP4 control	9.009808	MxWave is -1.4305%

Perplexity here measures observed-token likelihood under this exact windowing protocol. It is not a complete instruction-following or multimodal score.

Frozen no-thinking task screen

The diagnostic suite contained 82 cases, of which 69 had strict automatic scores. All four models saw identical prompts and decoding settings.

Table with columns: Model, Strict score
Model	Strict score
Public NVFP4 control	47/69 (68.12%)
BF16 source	46/69 (66.67%)
MxWave expert MXFP4 MSE	45/69 (65.22%)
MxWave expert MXFP4 RTN	45/69 (65.22%)

MxWave category results were 6/20 arithmetic, 5/15 logic, 15/15 structured output, 10/10 short retrieval, and 9/9 8K retrieval. There were no request errors.

Exact terminal next-token divergence from BF16

This screen retained the complete 248,320-token next-token distribution at one terminal position for each of 64 frozen contexts.

Table with columns: Metric, MxWave MSE, MxWave RTN, Public NVFP4
Metric	MxWave MSE	MxWave RTN	Public NVFP4
Mean forward KL	0.034157	0.031049	0.040895
Median forward KL	0.022420	0.015636	0.018090
P95 forward KL	0.093739	0.080812	0.122376
Mean reverse KL	0.034255	0.031715

Repeated serving measurements

Table with columns: Workload / metric, MxWave MXFP4, Public NVFP4, MxWave delta
Workload / metric	MxWave MXFP4	Public NVFP4	MxWave delta
Model-load memory	21.47 GiB	22.41 GiB	-0.94 GiB
8K prefill, C1 input throughput	6,596.38 tok/s	7,152.96 tok/s	-7.78%
8K prefill, C1 TTFT	1,240.51 ms	1,144.66 ms	+8.47%
32-to-512 decode, C1 output throughput	37.395 tok/s	36.440 tok/s

This is not a pure kernel comparison. MxWave ran weight-only MXFP4 W4A16 using Marlin, while the public NVFP4 control ran W4A4 using FlashInfer/CUTLASS. Throughput conclusions are workload-specific.

Size comparison

Table with columns: Checkpoint, Indexed tensor data, Difference
Checkpoint	Indexed tensor data	Difference
MxWave expert MXFP4 MSE	22,902,614,752 bytes (21.33 GiB)	—
Public NVFP4 control	23,909,493,472 bytes (22.27 GiB)	MxWave is 4.21% smaller

Reproduction

The current model-independent CLI was merged in MxWave PR #8. From MxWave commit 4309db0f639eefa3c3c8f126dba2bb89e15dd49e or later:

bash

mxwave-quantize-experts \  --model-dir /path/to/Nex-N2.5-mini-bf16 \  --output-dir /path/to/Nex-N2.5-mini-mxwave-mse \  --device cuda \  --method mse \  --scale-percentile 99.5 \  --mse-clip-depth 4 \  --tensor-row-chunk-size 2048 \  --host-tensor-cap-mib 1024 \  --source-repository nex-agi/Nex-N2.5-mini \  --source-revision 87420286149d9cce9bd46cd335ef9bda33c37c1b

Evaluated-artifact hashes

These hashes bind the artifact used for the numerical results above. The published config.json has a different hash solely because of the group-format normalization described above:

Table with columns: Artifact, SHA-256
Artifact	SHA-256
Ordered checkpoint content	`1d134e5d5c704cc76fe85268ee1f65ece0107e932e9de856c141bdb5fa9e8738`
`config.json`	`8ea02e392011595c2394a26862e51966608abab24fa2b750ce798b4ce67f6351`
Safetensors index	`69d4da6c13645e868454a7e22537e3f2ceef8883cf63817d67e2239145ad293c`
MxWave manifest	`29bd6b121e705e041f1e1520ce12d7e49b4fb4f07c66174907f6b63bd49a76c3`

Resume validation accepted all 87 completed shards and requantized zero shards.

Evaluation provenance

The compact machine-readable report and file manifest are included in evaluation/results.json and evaluation/SHA256SUMS.

Public control: primitive-ai/Nex-N2.5-mini-NVFP4 at revision ceaa4ce6d942c3d5456322d183b535a0c89a5fa6.
WikiText-2 dataset revision: b08601e04326c79dfdd32d625aee71d232d685c3.
PPL corpus SHA-256: 696cca6b65a171b0a358a4be6732cdfdf2dd6164a32e20fd70e3c13fc4dfae83.
PPL protocol SHA-256: ecc777cb86672560e95196d90e8bc63e2d5c2f7e9eaa9e8b90ba1babefdf91ed.
Exact-KL context manifest SHA-256: 9b72a2aa2b32c13dce3ff2e387d4f4a8d1383a240baf620a58d351566d8c9d3f.
Exact-KL protocol SHA-256: e37d5803afa723cd7505baf407476f36e86c3168e9fbaa69251289e672013f38.
Exact-KL combined comparison SHA-256: .

Limitations

This is expert-only quantization, so the remaining BF16 components limit the total size reduction.
The candidate is near BF16 on the measured PPL protocol, but it is not proven equivalent to BF16 across tasks or distributions.
The small frozen task screen and 64-position exact-KL screen are diagnostic, not broad capability benchmarks.
MSE scale search improved paired PPL over RTN but worsened the primary exact BF16-forward KL point estimate. The evidence is genuinely mixed.
Vision received only a smoke test. Image understanding and generation quality were not benchmarked.
Runtime results compare different activation-precision and kernel paths.
The checkpoint is validated for official vLLM 0.29 on Blackwell. It is not validated for Transformers inference, llama.cpp, Ollama, MLX, or LM Studio.
Only the Qwen3.5-MoE adapter and this checkpoint layout were validated. The engine is adapter-driven, but other architectures require their own adapter and independent evaluation.

License and attribution

This quantized derivative inherits the base model's Apache-2.0 license. Review the base model card for its intended use, restrictions, training information, and original citations.

Quantization and evaluation tooling: MxWave.