primitive-ai

Nanbeige4.3-33B-FP8

README

License: apache-2.0

Why this build

🎯 Drift +5.95×10⁻³ nats per token from BF16, top-1 agreement 97.9%. The NVFP4 build sits at +64.3 and 93.7%.
📏 Knowledge 89.3, tool calls 80.4, abstain 63.3; BF16 scores 89.7, 80.8 and 63.3. 99.1% of answers finish inside the budget.
⚡ 35.4 tok/s single-stream and 225 at concurrency 32 on one card, against 20.1 and 129 for BF16 on the same card.
🗜️ 35.53 GB, against 66.24 GB in BF16. At a 32,768-token window the KV pool is 262,448 tokens, against 113,648 for BF16.

Serve it

bash

hf download primitive-ai/Nanbeige4.3-33B-FP8 --local-dir ./Nanbeige4.3-33B-FP8 docker run --gpus '"device=0"' --ipc=host -p 8000:8000 -v $PWD:/models \  vllm/vllm-openai:v0.32.0 \  --model /models/Nanbeige4.3-33B-FP8 \  --max-model-len 131072 --gpu-memory-utilization 0.92 \  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

Checked with exactly this command on an idle card: boot 171 s, 33.44 GiB of weights, a 251,504-token KV pool at the 131,072-token window. What is 17*23? comes back as 391 with the reasoning split off by the parser, and as 391 with chat_template_kwargs: {"enable_thinking": false}; a weather question comes back as the parsed call get_weather {"city": "Lisbon"}. The model's window is 524,288 tokens; raise --max-model-len as far as the KV pool allows.

Measured

One RTX PRO 6000 Blackwell, 96 GB, no tensor parallelism, vLLM 0.32.0, BF16 KV cache. The 1,170-item knowledge suite and the 200-item tool-calling suite, temperature 0.6 / top_p 0.95 / top_k 20, thinking on, a 16,384-token budget, concurrency 32, auto-scored with no LLM judge. Throughput is 8K in / 512 out, prefix-cache free, two seeds per cell. KV pool is at a 32,768-token window and --gpu-memory-utilization 0.92.

Table with columns: build, format, size, drift (×10⁻³ nats/token), top-1 agreement, knowledge, call, abstain, finished, tok/s @1, tok/s @32, KV pool (tokens)
build	format	size	drift (×10⁻³ nats/token)	top-1 agreement	knowledge	call	abstain	finished	tok/s @1	tok/s @32	KV pool (tokens)
Nanbeige4.3-33B (BF16 release)	BF16	66.24 GB	0 (reference)	100%	89.7	80.8	63.3	99.2%	20.1	129	113,648
this repo	FP8 W8A8, every Linear	35.53 GB	+5.95	97.9%	89.3	80.4	63.3	99.1%	35.4	225	262,448
Mixed NVFP4 / FP8	NVFP4 W4A4, FP8 W8A8 in 216 modules	25.30 GB	+38.89	95.1%	88.9	79.0	63.3	99.1%	48.0	308	311,488
NVFP4	NVFP4 W4A4, every Linear	22.07 GB	+64.28	93.7%	89.0	78.8	64.2	99.1%	53.8	336	326,144

Drift and agreement come from 400 BF16 traces of the knowledge suite (215,950 completion tokens); the standard error over traces is 0.3 for this build. Tool calling is 3 runs per build; knowledge is 2 runs per quantized build and 1 for BF16.

What this table does not cover:

Context past 32K. The model's window is 524,288 tokens; we did not measure the quantized layers there.
Other GPUs. Only the RTX PRO 6000 Blackwell (sm_120) was tested.

Comparable with our other models

Accuracy numbers move for reasons that have nothing to do with the model: a shorter token budget, a different temperature, or whether the model was allowed to reason at all. So every number in this table, on this card and on our other cards, comes from the one fixed protocol described above, the same 1,370 items, auto-scored, no LLM judge.

Table with columns: model, shape, size, overall, knowledge, call, abstain, finished, out/answer
model	shape	size	overall	knowledge	call	abstain	finished	out/answer
Laguna-XS-2.1	31 B MoE	19.3 GiB	81.7	83.8	68.4	73.5	98.9%	1097
Nemotron-3.5-Lightning-30B-A3B	30 B MoE+Mamba	19.2 GiB	87.1	87.9	85.4	70.5	97.9%	1429
Ornith-1.5-35B-A3B	35 B MoE	22.6 GiB	88.7	91.7	74.4	60.0	99.3%	760
Muse-Glimmer-30B	30 B MoE	20.4 GiB	86.6	88.8	78.6	54.5	99.7%	800
Qwen3.8-27B	27 B dense	20.7 GiB	88.8	90.4	85.5	54.5	99.7%	651
Granite-4.2-30B	30 B dense	18.1 GB	85.5	86.2	85.8	60.8	98.5%	1502
Nex-N2.5-mini NVFP4	35 B MoE, 3 B active	23.91 GB	88.5	90.5	81.9	57.5	99.3%	524
Nex-N2.5-mini mixed	35 B MoE, 3 B active	26.04 GB	88.5	90.6	80.9	58.7	99.2%	504
Nex-N2.5-mini FP8	35 B MoE, 3 B active	38.13 GB	88.7	90.9	79.4	60.0	99.5%	545
K2-Horizon-MoVA-36B-A4B NVFP4	37 B MoE+MoVA, 4 B active	36.7 GB	84.1	86.5	71.8	60.4	95.6%	1234
K2-Horizon-MoVA-36B-A4B mixed	37 B MoE+MoVA, 4 B active	44.5 GB	84.9	87.3	73.5	58.9	96.3%	1118
Nemotron-3.5-Bolt-74B-A9B mixed	75 B MoE+Mamba, 9 B active	46.70 GB	89.3	90.6	87.9	58.3	98.5%	1173
Laguna-S-2.1	110 B MoE	64.0 GiB	84.3	87.1	64.6	81.0	97.3%	995
Qwen3.8-Flash-Next	180 B MoE, 6 B active	183.7 GB	90.3	92.2	84.8	56.7	99.5%	686
MiMo-V2.6-Flash-RL NVFP4	309 B MoE, 15 B active	182.40 GB	89.3	91.1	83.0	61.0	99.0%	664
Kolibri-1 FP8	78 B MoE, 3.5 B active	78.83 GB	85.0	86.1	86.1	48.0	97.0%	1824
Kolibri-1 NVFP4	78 B MoE, 3.5 B active	47.70 GB	84.9	86.2	84.9	48.0	96.4%	1947
Kolibri-1 mixed	78 B MoE, 3.5 B active	45.81 GB	83.9	85.0	85.5	47.5	96.6%	1848
LFM2.5-24B-A2B NVFP4A16	24 B MoE, 2.45 B active	14.95 GB	83.9	85.4	82.4	44.5	98.8%	1603
MiniMax-M3.1-Flash (release, 4 cards)	427 B MoE, 23 B active	250.12 GB	82.9	85.0	81.7	25.0	99.2%	513
MiniMax-M3.1-Flash-pruned-272B	272 B MoE, 23 B active	162.90 GB	82.5	84.4	80.4	32.5	98.8%	707
Nanbeige4.3-33B (BF16 release)	33 B dense	66.24 GB	87.9	89.7	80.8	63.3	99.2%	846
Nanbeige4.3-33B FP8 (this repo)	33 B dense	35.53 GB	87.5	89.3	80.4	63.3	99.1%	871
Nanbeige4.3-33B mixed	33 B dense	25.30 GB	87.0	88.9	79.0	63.3	99.1%	905
Nanbeige4.3-33B NVFP4	33 B dense	22.07 GB	87.1	89.0	78.8	64.2	99.1%	940

overall pools the two suites as 1,370 items, weighted 85.4% knowledge and 14.6% tool calling by item count. Read it with finished: overall scores an answer that overran the token budget as wrong, and cannot say whether the model needed the room or failed to stop. A gap under 1.0 in overall is a tie. Sizes are as each card reports them, which mixes GB and GiB.

What's in it

Nanbeige4.3-33B is a dense Llama-architecture model: 33.12 B parameters, 52 layers, hidden size 7168, 56 query and 8 KV heads, MLP width 22,016.

Table with columns: params, share
	params	share
MLP gate + up + down, 52 layers	24.62 B	74.3%
attention q + k + v, 52 layers	3.44 B	10.4%
attention o, 52 layers	2.67 B	8.1%
embeddings and LM head (untied)	2.39 B	7.2%

Table with columns: modules, count, format
modules	count	format
every decoder Linear (q, k, v, o, gate, up, down)	364	FP8 E4M3, one scale per output channel; activations FP8, one scale per token
embeddings, LM head, norms	all	BF16, byte-identical to the release

The checkpoint is compressed-tensors and runs on vLLM's stock kernels. q/k/v and gate/up share one format per layer because vLLM fuses them.

Licence

Apache-2.0, the base model's licence. This is a modified version of Nanbeige/Nanbeige4.3-33B: the decoder's linear layers were quantized and config.json carries a quantization_config. Everything else is unchanged.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

primitive-ai

Model Tree

Base

Nanbeige/Nanbeige4.3-33B

Quantized

this model

Input Modalities

Text

Output Modalities