GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AxionML
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
AxionML
Model Tree
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
Input Modalities
Output Modalities
Supported Functionality
| Architecture | Hybrid LatentMoE: Mamba-2 + MoE + attention, with MTP |
| Total Parameters | 550B |
| Activated Parameters | 55B |
| Context Length | Up to 1M tokens |
| Minimum GPUs | 4x B200 / 4x GB200 / 4x B300 / 4x GB300 / 8x H100 |
| Reasoning | On/off via chat template (enable_thinking) |
| Checkpoint Size | ~352 GB |
| Benchmark | BF16 | NVFP4 |
|---|---|---|
| Terminal-Bench 2.1 | 56.4 | 53.9 |
| SWE-Bench Verified | 70.7 | 69.5 |
| SWE-Bench Multilingual | 67.7 | 69.1 |
| GDPVal | 46.7 | 47.9 |
| TauBench V3 (avg) | 70.9 | 70.3 |
| BrowseComp | 44.4 | 41.4 |
| GPQA (no tools) | 87.0 | 87.9 |
| HLE (no tools) | 26.7 | 26.1 |
| SciCode (subtask) | 44.6 | 43.5 |
| IFBench (prompt) | 81.7 | 82.3 |
| AA-LCR | 65.4 | 65.5 |
| RULER 1M | 94.7 | 94.0 |
Scores reported by NVIDIA (NeMo Evaluator SDK). See the upstream card for the full table.
bash
python3 -m sglang.launch_server \ --model-path AxionML/Nemotron-3-Ultra-550B-A55B-NVFP4 \ --tp-size 4 \ --ep-size 4 \ --context-length 262144 \ --mem-fraction-static 0.85 \ --mamba-scheduler-strategy extra_buffer \ --mamba-backend flashinfer \ --attention-backend trtllm_mha \ --reasoning-parser nemotron_3 \ --tool-call-parser qwen3_coder \ --speculative-algorithm EAGLE \ --speculative-num-steps 5 \ --speculative-eagle-topk 1 \ --speculative-num-draft-tokens 6 \ --trust-remote-codebash
vllm serve AxionML/Nemotron-3-Ultra-550B-A55B-NVFP4 \ --trust-remote-code \ --tensor-parallel-size 4 \ --enable-expert-parallel \ --kv-cache-dtype fp8 \ --max-model-len 262144 \ --enable-chunked-prefill \ --enable-prefix-caching \ --reasoning-parser nemotron_v3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --mamba-ssm-cache-dtype float16 \ --mamba-backend flashinfer \ --speculative-config '{"method": "nemotron_h_mtp", "num_speculative_tokens": 5}'Tested upstream on 4x B200 with lmsysorg/sglang:v0.5.13 and vllm/vllm-openai:v0.22.0. For 1M context set SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 / VLLM_ALLOW_LONG_MAX_MODEL_LEN=1. With tools, send "chat_template_kwargs": {"enable_thinking": true, "force_nonempty_content": true}.
The base model was trained on data that may contain toxic language and societal biases. The quantized model inherits these limitations. It may generate inaccurate, biased, or offensive content. Please refer to the original model card and the upstream quantized model card for full details.
python3 -m sglang.launch_server \ --model-path AxionML/Nemotron-3-Ultra-550B-A55B-NVFP4 \ --tp-size 4 \ --ep-size 4 \ --context-length 262144 \ --mem-fraction-static 0.85 \ --mamba-scheduler-strategy extra_buffer \ --mamba-backend flashinfer \ --attention-backend trtllm_mha \ --reasoning-parser nemotron_3 \ --tool-call-parser qwen3_coder \ --speculative-algorithm EAGLE \ --speculative-num-steps 5 \ --speculative-eagle-topk 1 \ --speculative-num-draft-tokens 6 \ --trust-remote-codevllm serve AxionML/Nemotron-3-Ultra-550B-A55B-NVFP4 \ --trust-remote-code \ --tensor-parallel-size 4 \ --enable-expert-parallel \ --kv-cache-dtype fp8 \ --max-model-len 262144 \ --enable-chunked-prefill \ --enable-prefix-caching \ --reasoning-parser nemotron_v3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --mamba-ssm-cache-dtype float16 \ --mamba-backend flashinfer \ --speculative-config '{"method": "nemotron_h_mtp", "num_speculative_tokens": 5}'