Run it
vllm serve turintech/Nemotron-3.5-Bolt-E4B-Artemis-NVFP4 \ --dtype bfloat16 --max-model-len 4096 --language-model-only \ --enable-auto-tool-choice --tool-call-parser gemma4
Tested with vLLM 0.32.0rc1 (a development build) and, on a DGX Spark, vLLM 0.31.0. Every
result below was measured with thinking off; BFCL-v3 ran at a 65,536-token context and the other checks at 4,096. On a 16 GB GPU, add
--gpu-memory-utilization 0.92 --max-num-batched-tokens 2048
, and lower the first value if the GPU also drives a display; this is expected to
work, not tested on a 16 GB card. For images and audio, drop
--language-model-only; audio also needs
pip install "vllm[audio]".
curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{ "model": "turintech/Nemotron-3.5-Bolt-E4B-Artemis-NVFP4", "messages": [{"role": "user", "content": "In two sentences, why is the sky blue?"}], "chat_template_kwargs": {"enable_thinking": false}, "max_tokens": 256}'
The reply is in choices[0].message.content. Tool calls use the same endpoint with a tools list and come back in
choices[0].message.tool_calls.
Results

Table with columns: File, Size, KL chat (lower is closer), Generation, one request (tokens/s), Generation, 8 requests (tokens/s)| File | Size | KL chat (lower is closer) | Generation, one request (tokens/s) | Generation, 8 requests (tokens/s) |
|---|
| TurinTech NVFP4 | 11.24 GB | 0.063 | 202 | 1321 |
| caelunshun NVFP4 | 9.61 GB | 0.194 | 193 | 1366 |
| TurinTech FP8 | 12.69 GB | 0.010 | 183 | 1206 |
| BF16 original | 15.99 GB | reference | 139 | 844 |
Bold marks the best in each column. caelunshun's build is text only with an FP8 KV cache, which the KL measurement does not emulate, so its KL is a lower bound; tool calls and GSM8K are on the results page. One NVIDIA RTX 5090, vLLM 0.32.0rc1, thinking off. Full results, memory and method.
Files
Table with columns: File, Size, Use it for| File | Size | Use it for |
|---|
attn.safetensors | 1.17 GB | Unchanged tensors from NVIDIA's checkpoint, repackaged |
core.safetensors | 8.10 GB | Unchanged tensors from NVIDIA's checkpoint, repackaged |
mlp.safetensors | 1.86 GB | The language model's MLP weights, quantized |
model-frozen-shared-kv.safetensors | 110 MB | Redistributed unchanged from NVIDIA's checkpoint |
chat_template.jinja and 6 more | 32 MB | Configuration, tokenizer, chat template and weight index, which vLLM reads |
bias.md and 6 more | 30 MB | The original publisher's model-card documents, media and handler, unchanged |
LICENSE and 1 more | 5 kB | Licence and notice |
The checksums section lists every file. vllm serve turintech/Nemotron-3.5-Bolt-E4B-Artemis-NVFP4 downloads the 11 weight and configuration files, 11.28 GB in all.
Pick this model
- Good for Blackwell GPUs, including a DGX Spark, in 11.24 GB of weights with the vision and audio towers, with tool calls and GSM8K level with the original and closer to it than caelunshun's public NVFP4 build on all three texts.
- Pick another file when multi-turn tool calls matter: on BFCL-v3 they are 1.6 points below the original. Closeness is lower too: its KL is 5.0 to 6.1 times our FP8 checkpoint's. For text only, caelunshun's public NVFP4 build is 1.6 GB smaller and faster with eight requests at once, further from the original.
How it was made
Nemotron 3.5 Bolt E4B is NVIDIA's small chat model for text, images and audio, post-trained by NVIDIA from Google's
Gemma 4 E4B. This checkpoint comes from TurinTech's work with Artemis,
starting from NVIDIA's checkpoint
(nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16, revision
49843ee). Only the language model's MLP changed: its weights are NVFP4 and its activations stay BF16 (compressed-tensors NVFP4A16, GPTQ with llm-compressor). Attention, the embeddings, the
per-layer gates and projections and the vision and audio towers are NVIDIA's, unchanged. The 4-bit weights were calibrated on public text: WikiText-2 train, OpenAssistant train conversations, the CPython standard library and Berkeley Function Calling Leaderboard items. NVIDIA's
tokenizer, chat template, model-card documents and media are included unchanged.
How it was tested
Every number on this card was measured with vLLM 0.32.0rc1 on one NVIDIA RTX 5090, greedy decoding and thinking off,
against NVIDIA's BF16 original run the same way. Closeness is the KL divergence from the original on wiki, chat and
code text, computed in Hugging Face Transformers. Tasks are BFCL simple tool calls and the first 250 GSM8K
problems; the full BFCL-v3 and all 1,319 GSM8K problems were also run. Images and audio were only
spot-checked. The results page has every table, the method and memory.
Table with columns: File, Bytes, sha256| File | Bytes | sha256 |
|---|
LICENSE | 2,619 | 2ab44b68365473c112f5092211a38f231cb23e50de68b75a13369adbd76a74df |
NOTICE | 2,002 | 601ba361d281a876fcf26f8e2fe7b1767adde097646937d551a3644a75b29ca4 |
attn.safetensors | 1,174,422,384 | d13c9218f93f63b5cfc1416c51b6382fccca5ee4648fc7527893071cdbcdb69b |
bias.md | 3,254 | bbe310672de023a587ba92e5c1d5da2edc22955f8822b2f3ffe00afca2dd8de1 |
bolt-smart-and-fast.png | 444,073 | 244b8a0f465c13fbfa81c8046b0bd4471d2202c275ea950c4845374788852145 |
chat_template.jinja | 20,161 | 35b7d0641b6fb4ee41c3bc623e85bdb7d583c40f6ae9f24514ae58a9d57119f3 |
config.json | 5,877 | 155834599e318b5a4d384e6f543608573a1fdcd1029042846f8dd02626e4ce46 |
core.safetensors | 8,102,007,484 | 899fa208df63ab0058dab6dc75f3489967bf1535cd211423a151d2c1f762e88a |
explainability.md | 2,674 | 7671d31882b60bb4eb385e324717688941fb6d1ed99359af7b1603c98018bac5 |
generation_config.json | 208 | d4226bbe3117d2d253ba4609720ba82c6c4ce4627a9a6ae05387c78983ac03de |
handler.py | 7,165 | 48adddae3baf5adfb2cca8412df3c312b9ddc4550146708a53ac77945d4d66be |
mlp.safetensors | 1,857,994,928 | 029b5e99c85bc175c0e608bc7ec8833383706c7941f5dbe461bc76e7148f940a |
model-frozen-shared-kv.safetensors | 110,118,048 | 32aaa38402af5a51c6d6b27aeb8a26d3e6964e5590330f0e7b2f546608da4023 |
model.safetensors.index.json | 202,362 | c76644be7a62c1896e7b3a0ea6718ebb8c8a10c94bf7ec0eaaf076e1f1eac4d1 |
privacy.md | 1,671 | c048eb257c10d128d35a8a90d8c98e7684e2019d9e110f7db1543cf88f1ce63a |
processor_config.json | 1,689 | 32bdf45d2ad4cc29a0822ddd157a182de76644f0419a6228d151495256e9813c |
reachy-bolt.gif | 29,796,429 | 446ab42f5a52e2d9d2bb09a76efa4439d96929090eaab0f3d2fce4f9e330cd75 |
safety.md | 2,292 | 50df88aa4e1f25c3504522ed11738075f2d97150c15a89508a1eb6912aec22f0 |
tokenizer.json | 32,169,626 | cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f |
tokenizer_config.json | 23,725 | b3da1b7227c5d35fd905ad13200a08be63cb346b2d1bd4f6d58878c9783d5de1 |
Thanks and feedback
Thanks to the vLLM and llm-compressor projects, to caelunshun for the public NVFP4 build this card compares with, and to the
open-source community, for pushing the limits of serving models efficiently. Their work was our foundation
throughout, and this checkpoint builds on theirs.
This checkpoint is new for us, and feedback is welcome: whether it runs well on a given GPU, anything that did not
work, or a benchmark that should be measured differently. Please open a discussion on this page.
Artemis is TurinTech's optimisation platform: teams use it to make their own models smaller and faster, or to optimise their code.
All our models · LinkedIn · Questions and requests
OpenMDW-1.1, as released by NVIDIA (licence); see LICENSE and NOTICE. An independent TurinTech release, not affiliated with or endorsed by NVIDIA. When serving this checkpoint to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.
These files keep improving.