GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
soyrsoyr
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
soyrsoyr
Model Tree
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
Input Modalities
Output Modalities
Supported Functionality
87347881.config.json, recipe.yaml when present, and pr3118-validation.json. NVFP4A16 is weight-only FP4 with 16-bit activations, not calibrated NVFP4 W4A4. MXFP4 uses dynamic activation quantization.Validated runtime baseline: vllm==0.29.1rc1.dev79+g767d1c4d4, Transformers , CUDA 13.0. MXFP4 requires your B200 run to establish runtime compatibility.
5.17.0bash
vllm serve soyrsoyr/Nemotron3.5-Lightning-30B-A3B-FP8Dyn-MTP-MXFP4-FromFP8MTP-pr3118-validation --dtype bfloat16 --max-model-len 1024 --enforce-eager --gpu-memory-utilization 0.85 --speculative-config '{"method":"mtp","num_speculative_tokens":1}' --limit-mm-per-prompt '{"image":0,"video":0}'
After downloading this snapshot, python verify_mtp.py /path/to/snapshot runs two prompts and requires positive draft-token metrics. A successful model load alone is not counted as an MTP pass. The script records generations and speculative decoding metrics.