GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
AxionML
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
AxionML
Model Tree
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
Input Modalities
Output Modalities
Supported Functionality
| Architecture | Hybrid MoE: Mamba-2 + MoE + attention, with MTP |
| Total Parameters | 30B |
| Activated Parameters | 3B |
| Context Length | Up to 1M tokens |
| Single-GPU Deployment | 1x DGX Spark (GB10), 1x H100, RTX 5090 |
| Speculative Decoding | MTP (built in), plus separate DSpark and DFlash drafters |
| Checkpoint Size | ~22 GB |
| Benchmark | BF16 | NVFP4 |
|---|---|---|
| MMLU Pro | 81.94 | 81.62 |
| GPQA Diamond (no tools) | 75.44 | 75.57 |
| HLE (text-only) | 11.72 | 10.47 |
| SciCode | 32.60 | 31.38 |
| SWE-bench Verified | 51.56 | 52.80 |
| SWE-bench Multilingual | 39.33 |
Scores reported by NVIDIA (NeMo Gym / NeMo Evaluator). Recommended sampling:
temperature=1.0,top_p=0.95.
in_proj / out_projbash
sglang serve \ --model-path AxionML/Nemotron-3.5-Lightning-30B-A3B-NVFP4 \ --mamba-backend flashinfer \ --mamba-ssm-dtype float16 \ --enable-mamba-cache-stochastic-rounding \ --mamba-cache-philox-rounds 5 \ --mem-fraction-static 0.85 \ --cuda-graph-max-bs-decode 16 \ --reasoning-parser nemotron_3 \ --tool-call-parser qwen3_coderbash
vllm serve AxionML/Nemotron-3.5-Lightning-30B-A3B-NVFP4 \ --moe-backend marlin \ --kv-cache-dtype fp8 \ --enable-prefix-caching \ --gpu-memory-utilization 0.85 \ --mamba-backend flashinfer \ --mamba-cache-mode align \ --reasoning-parser nemotron_v3 \ --speculative_config.model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark \ --speculative_config.num_speculative_tokens 3 \ --tool-call-parser qwen3_coder \ --enable-auto-tool-choiceSGLang command is the upstream 1x B200 recipe (lmsysorg/sglang:dev-nemotron3-5-lightning); the vLLM command is the upstream DGX Spark recipe with DSpark (vllm/vllm-openai:v0.27.1). For MTP in SGLang add --speculative-algorithm EAGLE --speculative-draft-model-path AxionML/Nemotron-3.5-Lightning-30B-A3B-NVFP4 --speculative-num-steps 5 --speculative-eagle-topk 1 --speculative-num-draft-tokens 6.
The base model was trained on data that may contain toxic language and societal biases. The quantized model inherits these limitations. It may generate inaccurate, biased, or offensive content. Please refer to the original model card and the upstream quantized model card for full details.
| 36.47 |
| Terminal-Bench 2.1 | 24.58 | 23.46 |
| PinchBench | 85.37 | 83.43 |
| BrowseComp | 36.97 | 36.81 |
| IFBench (loose) | 71.88 | 72.88 |
| AA-LCR | 52.00 | 49.19 |