nex-agi

Nex-N2-Mini-NVFP4

README

License: apache-2.0

Quantization Details

The model is quantized with compressed-tensors (quant_method: compressed-tensors, format: mixed-precision). The full recipe ships in recipe.yaml.

NVFP4 — W4A4 (`nvfp4-pack-quantized`)

Applied to the MoE feed-forward stack in layers 0–31 — this is the bulk of the parameters:

model.language_model.layers.{0..31}.mlp.experts.{0..255}.{gate,up,down}_proj
model.language_model.layers.{0..31}.mlp.shared_expert.{gate,up,down}_proj

Table with columns: Weights, Activations
	Weights	Activations
Format	FP4 (E2M1), packed 2-per-byte	FP4 (E2M1)
Strategy	`tensor_group`, group size 16	`tensor_group`, group size 16
Block scale dtype	`float8_e4m3`	`float8_e4m3`
Calibration	static, `memoryless_minmax`	dynamic (`local`), `static_minmax` global scale

FP8 — W8A8 (`float-quantized`)

Applied to attention, the linear-attention projections, the LM head, and the MoE stack of the last 8 layers (32–39), which are the most quantization-sensitive:

self_attn.{q,k,v,o}_proj
linear_attn.{in_proj_qkv,in_proj_z,out_proj}
lm_head
layers.{32..39}.mlp.experts.*.{gate,up,down}_proj and layers.{32..39}.mlp.shared_expert.{gate,up,down}_proj

Table with columns: Weights, Activations
	Weights	Activations
Format	FP8 (E4M3)	FP8 (E4M3)
Strategy	per-`channel`, symmetric	per-`token`, symmetric
Calibration	static, `memoryless_minmax`	dynamic

Kept in BF16

The full vision tower (model.visual.*, 333 tensors)
MoE routers — mlp.gate, mlp.shared_expert_gate
Linear-attention gating / state projections — linear_attn.in_proj_a, linear_attn.in_proj_b, linear_attn.conv1d, A_log, dt_bias, and all linear_attn.norm*
Token embeddings and all RMSNorm weights

The KV cache is not quantized (kv_cache_scheme: null).

Hardware

NVFP4 reaches its full speedup on NVIDIA Blackwell GPUs (SM100 / SM120 — B200, GB200, RTX PRO 6000 Blackwell), which have native FP4 tensor cores. vLLM can still load and run this checkpoint on pre-Blackwell hardware through its NVFP4 dequantization/emulation path, in which case you get the memory savings but not the throughput gain.

At 24.8 GB of weights the model serves on a single 80 GB card with room for a long-context KV cache; the launch command below is tuned for that shape.

Usage

Deployment with vLLM

Note: Qwen3.5-MoE support, the NVFP4 compressed-tensors kernels, and the --language-model-only flag all require a recent vLLM build. Use a current release or nightly.

bash
vllm serve /models/Nex-N2-Mini-NVFP4 \
  --served-model-name nex-n2 \
  --host 0.0.0.0 --port 8000 \
  --trust-remote-code --language-model-only \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --max-num-seqs 32 \
  --gpu-memory-utilization 0.60 \
  --max-num-batched-tokens 8192 \
  --enable-chunked-prefill

Flag notes:

--language-model-only skips the (unquantized, BF16) vision tower and serves the model as a pure text LLM. Drop it if you want the multimodal path.
--max-model-len 131072 is a conservative cap; the checkpoint's max_position_embeddings is 262144.
--gpu-memory-utilization 0.60 and --max-num-seqs 32 leave generous headroom — raise them if you have spare VRAM and want more concurrency.
--tensor-parallel-size 1 is sufficient at this precision. Increase it for extra KV-cache capacity, not for weight capacity.

Query it through the OpenAI-compatible API:

bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nex-n2",
    "messages": [{"role": "user", "content": "Write a Python function that merges overlapping intervals."}],
    "temperature": 0.7,
    "top_p": 0.95,
    "top_k": 40
  }'

Function Calling and Reasoning Parser

Nex-series models emit explicit reasoning traces and support robust function calling. To have vLLM split the reasoning trace from the final answer and parse tool calls into the OpenAI tool_calls schema, add:

bash
--reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Without these flags the reasoning trace and the <tool_call> blocks are returned inline in content, and you parse them yourself.

Recommended Sampling Parameters

temperature: 0.7
top_p: 0.95
top_k: 40

Deployment with SGLang

The base model is best served with the customized sglang fork described in the Nex-N2-mini card. That fork does not currently consume compressed-tensors NVFP4 checkpoints — use the BF16 base model there, and this repository with vLLM.

Performance

The scores below are the published results for the BF16 base models and are reproduced here for reference. They were not re-measured on this NVFP4 checkpoint; expect a small degradation typical of 4-bit post-training quantization. If you benchmark it, we would welcome the numbers in the Community tab.

Table with columns: Benchmark, Nex-N2-mini, Nex-N2-Pro, GPT-5.5, Opus 4.7, Kimi-K2.6, GLM-5.1, MiniMax M3, DeepSeek-V4-Pro
Benchmark	Nex-N2-mini	Nex-N2-Pro	GPT-5.5	Opus 4.7	Kimi-K2.6	GLM-5.1	MiniMax M3	DeepSeek-V4-Pro
Agent
BrowseComp	74.1

Nex-N2 Benchmark Overview

About Nex-N2

Nex-N2 is an agent model built for real-world productivity scenarios. With first-tier coding and agentic capabilities, it keeps driving complex, long-horizon tasks forward in real environments to deliver stable, end-to-end results.

Rather than treating reasoning, tool use, and environment execution as separate capabilities, Nex-N2 unifies them through an Agentic Thinking framework that connects requirement understanding, task planning, code implementation, environmental feedback, evaluation and debugging, and continuous iteration into a single closed loop. The framework has two parts:

Adaptive Thinking lets the model decide on its own when to think and how deeply — executing simple actions quickly while reasoning thoroughly on critical decisions.
Coherent Thinking carries one consistent reasoning paradigm across general reasoning and diverse agentic tasks, staying consistent across tasks and modalities to enable stable capability transfer.

Nex-N2 ships in two variants, both post-trained on the Qwen3.5 series: Nex-N2-Pro (built on Qwen3.5-397B-A17B) and Nex-N2-mini (built on Qwen3.5-35B-A3B-Base).

Model Architecture

Table

Architecture	`Qwen3_5MoeForConditionalGeneration` (hybrid linear + full attention MoE)
Layers	40 (full attention every 4th layer, linear attention elsewhere)
Hidden size	2048
Experts	256, 8 active per token, plus 1 shared expert
Context length	262,144
Vocabulary	248,320

Nex-N2-Pro: Hugging Face | ModelScope
Nex-N2-mini (BF16 base): Hugging Face | ModelScope

License

Released under the Apache 2.0 license, matching the base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

nex-agi

Model Tree

Base

nex-agi/Nex-N2-mini

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Quantization Details

The model is quantized with compressed-tensors (quant_method: compressed-tensors, format: mixed-precision). The full recipe ships in recipe.yaml.

NVFP4 — W4A4 (`nvfp4-pack-quantized`)

Applied to the MoE feed-forward stack in layers 0–31 — this is the bulk of the parameters:

model.language_model.layers.{0..31}.mlp.experts.{0..255}.{gate,up,down}_proj
model.language_model.layers.{0..31}.mlp.shared_expert.{gate,up,down}_proj

Table with columns: Weights, Activations
	Weights	Activations
Format	FP4 (E2M1), packed 2-per-byte	FP4 (E2M1)
Strategy	`tensor_group`, group size 16	`tensor_group`, group size 16
Block scale dtype	`float8_e4m3`	`float8_e4m3`
Calibration	static, `memoryless_minmax`	dynamic (`local`), `static_minmax` global scale

FP8 — W8A8 (`float-quantized`)

Applied to attention, the linear-attention projections, the LM head, and the MoE stack of the last 8 layers (32–39), which are the most quantization-sensitive:

self_attn.{q,k,v,o}_proj
linear_attn.{in_proj_qkv,in_proj_z,out_proj}
lm_head
layers.{32..39}.mlp.experts.*.{gate,up,down}_proj and layers.{32..39}.mlp.shared_expert.{gate,up,down}_proj

Table with columns: Weights, Activations
	Weights	Activations
Format	FP8 (E4M3)	FP8 (E4M3)
Strategy	per-`channel`, symmetric	per-`token`, symmetric
Calibration	static, `memoryless_minmax`	dynamic

Kept in BF16

The full vision tower (model.visual.*, 333 tensors)
MoE routers — mlp.gate, mlp.shared_expert_gate
Linear-attention gating / state projections — linear_attn.in_proj_a, linear_attn.in_proj_b, linear_attn.conv1d, A_log, dt_bias, and all linear_attn.norm*
Token embeddings and all RMSNorm weights

The KV cache is not quantized (kv_cache_scheme: null).

Hardware

At 24.8 GB of weights the model serves on a single 80 GB card with room for a long-context KV cache; the launch command below is tuned for that shape.

Usage

Deployment with vLLM

Note: Qwen3.5-MoE support, the NVFP4 compressed-tensors kernels, and the --language-model-only flag all require a recent vLLM build. Use a current release or nightly.

bash
vllm serve /models/Nex-N2-Mini-NVFP4 \
  --served-model-name nex-n2 \
  --host 0.0.0.0 --port 8000 \
  --trust-remote-code --language-model-only \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --max-num-seqs 32 \
  --gpu-memory-utilization 0.60 \
  --max-num-batched-tokens 8192 \
  --enable-chunked-prefill

Flag notes:

--language-model-only skips the (unquantized, BF16) vision tower and serves the model as a pure text LLM. Drop it if you want the multimodal path.
--max-model-len 131072 is a conservative cap; the checkpoint's max_position_embeddings is 262144.
--gpu-memory-utilization 0.60 and --max-num-seqs 32 leave generous headroom — raise them if you have spare VRAM and want more concurrency.
--tensor-parallel-size 1 is sufficient at this precision. Increase it for extra KV-cache capacity, not for weight capacity.

Query it through the OpenAI-compatible API:

bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nex-n2",
    "messages": [{"role": "user", "content": "Write a Python function that merges overlapping intervals."}],
    "temperature": 0.7,
    "top_p": 0.95,
    "top_k": 40
  }'

Function Calling and Reasoning Parser

bash
--reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Without these flags the reasoning trace and the <tool_call> blocks are returned inline in content, and you parse them yourself.

Recommended Sampling Parameters

temperature: 0.7
top_p: 0.95
top_k: 40

Deployment with SGLang

Performance

Table with columns: Benchmark, Nex-N2-mini, Nex-N2-Pro, GPT-5.5, Opus 4.7, Kimi-K2.6, GLM-5.1, MiniMax M3, DeepSeek-V4-Pro
Benchmark	Nex-N2-mini	Nex-N2-Pro	GPT-5.5	Opus 4.7	Kimi-K2.6	GLM-5.1	MiniMax M3	DeepSeek-V4-Pro
Agent
BrowseComp	74.1

Nex-N2 Benchmark Overview

About Nex-N2

Adaptive Thinking lets the model decide on its own when to think and how deeply — executing simple actions quickly while reasoning thoroughly on critical decisions.
Coherent Thinking carries one consistent reasoning paradigm across general reasoning and diverse agentic tasks, staying consistent across tasks and modalities to enable stable capability transfer.

Nex-N2 ships in two variants, both post-trained on the Qwen3.5 series: Nex-N2-Pro (built on Qwen3.5-397B-A17B) and Nex-N2-mini (built on Qwen3.5-35B-A3B-Base).

Model Architecture

Table

Architecture	`Qwen3_5MoeForConditionalGeneration` (hybrid linear + full attention MoE)
Layers	40 (full attention every 4th layer, linear attention elsewhere)
Hidden size	2048
Experts	256, 8 active per token, plus 1 shared expert
Context length	262,144
Vocabulary	248,320

Nex-N2-Pro: Hugging Face | ModelScope
Nex-N2-mini (BF16 base): Hugging Face | ModelScope

License

Released under the Apache 2.0 license, matching the base model.

Nex-N2-Mini-NVFP4

README

Quantization Details

NVFP4 — W4A4 (nvfp4-pack-quantized)

FP8 — W8A8 (float-quantized)

Kept in BF16

Hardware

Usage

Deployment with vLLM

Function Calling and Reasoning Parser

Recommended Sampling Parameters

Deployment with SGLang

Performance

About Nex-N2

Model Architecture

Related Models

License

Explore FriendliAI today

README

Quantization Details

NVFP4 — W4A4 (nvfp4-pack-quantized)

FP8 — W8A8 (float-quantized)

Kept in BF16

Hardware

Usage

Deployment with vLLM

Function Calling and Reasoning Parser

Recommended Sampling Parameters

Deployment with SGLang

Performance

About Nex-N2

Model Architecture

Related Models

License

NVFP4 — W4A4 (`nvfp4-pack-quantized`)

FP8 — W8A8 (`float-quantized`)

NVFP4 — W4A4 (`nvfp4-pack-quantized`)

FP8 — W8A8 (`float-quantized`)