abacusai

Smaug-Nemotron-3.5-NVFP4

README

License: openmdw-1.1

1. Model Introduction

Smaug-Nemotron-3.5-NVFP4 is the NVFP4 release of Smaug-Nemotron-3.5, Abacus.AI's agentic fine-tune of NVIDIA Nemotron 3.5 Super VL 120B-A12B. It is fine-tuned directly from NVIDIA Nemotron 3.5 Super VL 120B-A12B NVFP4, a 120B-parameter (12B active) hybrid Mamba-2 / MoE / Attention vision-language model quantized to NVFP4. It improves on the NVFP4 base on LiveBench 2.0 (+10.9 on Coding, +8.1 on Agentic and +5.2 overall, release 2026-10-14) and on agentic coding (+3.0 on DeepSWE, +2.0 on SWE-bench Multilingual, +1.9 on NL2RepoBench, +1.2 on Terminal-Bench 2.1), and holds reasoning, coding, instruction-following, long-context and vision scores at parity with the base (GPQA Diamond, LiveCodeBench v6, SciCode, IFBench, AA-LCR and V*).

We release the model in NVFP4 because that is where it is cheapest to run: 4-bit weights put the full 120B model in about 75 GB, roughly a third of its BF16 size, so it fits on a single H200, two H100 or one GB200. Carrying the agentic gains into 4 bits took more than quantizing a BF16 fine-tune afterwards, which would erase a change this small. Instead we fine-tuned the NVFP4 checkpoint itself, with a loss designed for it: the model improves through better actions, its thinking is held close to the base, and every training step runs the exact NVFP4 weights that are served, so what the model learns survives quantization intact (see Section 4). Everything else is exactly NVIDIA's NVFP4 checkpoint, and any stack that serves Nemotron 3.5 Super VL NVFP4 serves this model as a drop-in replacement.

This card describes the training approach and the evaluation results. Training data consists of filtered multi-turn, tool-using coding trajectories.

2. Model Summary

Table

Base Model	LLM_MARKDOWN_PROTECTED_0
BF16 Sibling	LLM_MARKDOWN_PROTECTED_1
Total Parameters	120B (12B active)
Architecture	Mamba-2 + MoE + Attention hybrid with a C-RADIO vision encoder and an MTP head (unchanged)
Inputs / Outputs	Text, images and video / text (as the base)
Precision	NVFP4 mixed precision (as the base): vision encoder, connector, Mamba convolutions and embeddings in BF16, MoE router in FP32
Context Length	Up to 1M tokens (as the base)
Reasoning Mode	On by default; configurable via chat template (`enable_thinking=True/False`)
Recommended Sampling	Temperature 1.0, Top_P 0.95 (vision: temperature 0.6, as the base)
Adaptation	Supervised fine-tuning on agentic trajectories
Training Hardware	One 8× NVIDIA B300 node
License	OpenMDW License Agreement, version 1.1 (inherited)

3. Evaluation

Smaug-Nemotron-3.5-NVFP4 vs its base

Higher is better. Base (NVFP4) and BF16 scores from the Nemotron 3.5 Super VL NVFP4 model card, except NL2RepoBench (our runs) and LiveBench (from livebench.ai).

All results were produced with vLLM serving the NVFP4 weights (FP8 KV cache as set in the checkpoint, no MTP speculative decoding) on NVIDIA B300 GPUs, one GPU per replica, temperature = 1.0, top_p = 0.95 (AA-LCR: top_p 1.0, as in NVIDIA's recipe; V*: temperature 0.6, the base's vision setting), thinking enabled, following the NVIDIA Nemotron 3.5 Super reproducibility recipes wherever one exists.

SWE-bench Verified: NVIDIA NeMo Gym recipe (OpenCode agent), average of 2 runs over all 500 tasks.
SWE-bench Multilingual: NVIDIA NeMo Gym recipe (OpenCode agent), average of 2 runs over all 300 tasks, scored with NVIDIA's grader as is.
Terminal-Bench 2.1: NVIDIA NeMo Gym recipe with the Terminus 2 agent, average of 3 runs over all 89 tasks at NVIDIA's 262,144-token context, timeouts counted as failures, with past reasoning kept in the conversation history (see Section 7).
DeepSWE: NVIDIA NeMo Gym recipe (OpenCode agent), 113 tasks, single run; served with vLLM on NVIDIA B200 GPUs (tensor parallel 2) with the same sampling and chat-template settings.
NL2RepoBench: the benchmark's official OpenHands 0.56 scaffold (NL2RepoBench repository at commit 781a1da), 104 repositories, mean fraction of hidden tests passed, single run; NVIDIA publishes no score, so the base (NVFP4) and BF16 columns are our runs under the same protocol (B200, tensor parallel 2).
LiveBench 2.0: scores for all three models as published on the LiveBench 2.0 leaderboard, release 2026-10-14 (last updated 2026-10-25), one run per suite. Overall is the mean of the seven category scores; Agentic is the Long Agentic category and Coding the Coding category.
LiveCodeBench v6: official recipe, 454 problems, pass@1, average of 2 runs.
SciCode: official recipe, subtask accuracy (65 problems, 288 sub-steps), average of 2 runs.
AA-LCR v1.1: official recipe, 100 questions, average of 4 runs.
GPQA Diamond: official recipe, 198 questions × 8 samples, single run.
IFBench: official recipe, 300 prompts × 8 samples, strict reward (NVIDIA's metric), single run.
V*: NVIDIA publishes no recipe, so we used our own harness following the protocol in NVIDIA's technical report: static multiple choice without a visual-search tool, full-resolution images, answers matched by letter with a GPT-3.5-Turbo fallback extractor; 191 questions, average of 4 runs.

4. Training Approach

Smaug-Nemotron-3.5-NVFP4 was fine-tuned from the NVFP4 base model with the same goal and data recipe as Smaug-Nemotron-3.5: to improve agentic coding, tool usage, and long-horizon multi-turn work through better actions, using curated multi-turn, tool-using trajectories with the model's reasoning kept in the conversation history. During training the model learns only from its actions, while the loss design holds its thinking close to the base model. We also keep every change tiny and surgical, so that the model's general capabilities do not degrade.

That requirement exposes a problem with standard fine-tuning. When a small learned update is combined with the base weights and rounded into the shipped format, most of it simply vanishes, in BF16 and even more so in NVFP4. To solve this, we use merge-aware training, which changes how the loss is computed: every training step runs the exact weights that will be served, in the exact number format, so nothing learned is lost at release. The final model moves only very slightly from the base. That is enough for clear agentic gains, while reasoning, coding, instruction-following, long-context and vision scores stay at parity. Training ran on a single 8× NVIDIA B300 node.

A loss built for the NVFP4 release

To make fine-tuning work for the NVFP4 release, we redesigned how the loss is computed: the loss always sees the model exactly as it will be served.

Train on the served weights. Each adapted layer has a base weight \(W\) and a small low-rank update \(sBA\), with factors \(A\) and \(B\) and a fixed scale \(s\). Every forward pass uses the weight the release will contain, rounded by \(Q_{\mathrm{fmt}}\) to the layer's own format (BF16, or FP8 with NVIDIA's per-tensor scale kept frozen):

W' = Q fmt ​ (W + s B A)

Rounding has no gradient, so for a layer input \(x\) each layer adds a correction built from its low-rank path \(\ell(x)\): it is zero in value but carries the low-rank gradient, with \(\mathrm{sg}\) the stop-gradient:

h = x W' ⊤ + (ℓ (x) - sg [ℓ (x)]), ℓ (x) = s \cdot sg [x] A ⊤ B ⊤

The layer output \(h\) equals \(x,W'^{\top}\), exactly what the release serves, while \(A\) and \(B\), the trainable parameters \(\theta\), train in full precision: progress too small to change a stored value builds up until it crosses a rounding boundary, and only then does the loss see it. Layers in coarser formats get larger learning rates. The trained model \(\pi_\theta\) is the whole network run this way, with the serving engine's numerics (FP8 KV cache, FP8 and NVFP4 activations, and 4-bit experts); the reference model \(\pi_{\mathrm{ref}}\) is the same network with \(B = 0\), the base model.

Learn from actions, hold the thinking in place.

L (θ) = - ∣ A ∣ 1 ​ t \in A \sum ​ lo g π θ ​ (y t ​ ∣ y < t ​) + β ∣ T ∣ 1 ​ r \sum ​ ∣ T r ​ ∣ k 3 ​ (R r ​)

The loss \(\mathcal{L}(\theta)\) has two terms:

Actions: the usual next-token loss on each token \(y_t\) given the earlier tokens \(y_{\lt t}\), applied only to the action tokens \(\mathcal{A}\). Thinking tokens get no likelihood term of their own.
Thinking: a penalty on drift in the reasoning. For each trajectory \(r\), \(R_r\) is the average gap \(\log \pi_{\mathrm{ref}} - \log \pi_\theta\) over its thinking tokens \(\mathcal{T}_r\); it is zero when the model on average rates that reasoning as the base does. The penalty \(k_3(R_r)\), with \(k_3(R) = e^{R} - R - 1\), is never negative, zero only at \(R_r = 0\) and close to \(R_r^2/2\) for small drift. Trajectories count in proportion to their share of all thinking tokens \(\mathcal{T}\), and \(\beta\) sets the penalty's strength.

Exact by construction. Training starts from the base model itself: with \(B = 0\), \(W' = W\) and the thinking penalty is exactly zero. And what ships is what was trained: the released checkpoint stores exactly the \(W'\) the loss was computed on.

Only the language model was adapted; the vision encoder, the vision-language connector and the MTP head are identical to the base. No architectural changes were made, and the NVFP4 quantization layout matches NVIDIA's NVFP4 release.

5. General Capabilities

Reasoning, coding, instruction-following, long-context and vision benchmarks (GPQA Diamond, LiveCodeBench v6, SciCode, IFBench, AA-LCR, V*) are at parity with the NVFP4 base.

6. Deployment

[!Note] Because the architecture and quantization are unchanged, Smaug-Nemotron-3.5-NVFP4 runs anywhere the Nemotron 3.5 Super VL NVFP4 checkpoint runs. Use the base model's deployment recipes (vLLM, SGLang, TensorRT-LLM) with MODEL=abacusai/Smaug-Nemotron-3.5-NVFP4.

2× H100 with vLLM (vLLM version: vllm/vllm-openai:v0.32.0), up to 1M context; for one H200, use --tensor-parallel-size 1:

shell

export MODEL=abacusai/Smaug-Nemotron-3.5-NVFP4vllm serve $MODEL \  --served-model-name Smaug-Nemotron-3.5-NVFP4 \  --tensor-parallel-size 2 \  --max-num-seqs 128 \  --trust-remote-code \  --gpu-memory-utilization 0.89 \  --max-num-batched-tokens 16384 \  --block-size 64 \  --prefix-match-unit 64 \  --compilation_config.max_cudagraph_capture_size 256 \  --mamba-backend flashinfer \  --mamba-ssm-cache-dtype float16 \  --enable-mamba-cache-stochastic-rounding \  --mamba-cache-philox-rounds 5 \  --enable-mamba-shared-prefix-checkpoint \  --mamba-cache-mode align \  --speculative-config.method mtp \  --speculative-config.num_speculative_tokens 3 \  --video-pruning-rate 0.5 \  --media-io-kwargs '{"video": {"fps": 2, "num_frames": 256}}' \  --enable-auto-tool-choice \  --tool-call-parser qwen3_coder \  --reasoning-parser nemotron_v3 \  --default-chat-template-kwargs '{"truncate_history_thinking": false}'

Our evaluations ran on vLLM 0.32 with one B300 per replica, the checkpoint's FP8 KV cache and without MTP.

7. Model Usage

Sampling behaviour is inherited from the base: temperature 1.0, top_p 0.95, the settings used for every number above (AA-LCR: top_p 1.0, as in NVIDIA's recipe). For image and video input, use the base's vision settings, temperature 0.6, top_p 0.95, as for V* above. For reasoning on/off, image, video, streaming and tool-calling examples, see the base model's API client section; all of them apply unchanged with model = "Smaug-Nemotron-3.5-NVFP4" (the served model name above).

Coding Agent Framework

The model is trained for multi-turn tool use and works with agent frameworks that speak the OpenAI chat-completions contract.

For multi-turn conversations and tool calls, pass the complete assistant message back to messages, including the reasoning field and tool_calls, not just content. The chat template drops past reasoning by default; serve with truncate_history_thinking: false (as above) to keep it.

8. License

Smaug-Nemotron-3.5-NVFP4 is a derivative of NVIDIA Nemotron 3.5 Super VL 120B-A12B NVFP4 and is released under the OpenMDW License Agreement, version 1.1, inherited from the base model. For the base model's bias, explainability, privacy and safety documentation, see the base model card.

This model is not affiliated with or endorsed by NVIDIA.

9. Citation

bibtex

@misc{abacusai2026smaugnemotron35nvfp4,  title  = {Smaug-Nemotron-3.5-NVFP4},  author = {Abacus.AI},  year   = {2026},  note   = {Agentic supervised fine-tune of nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4},  url    = {https://huggingface.co/abacusai/Smaug-Nemotron-3.5-NVFP4}}

10. Contact Us

If you have any questions, please reach out at Abacus.AI.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider