GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
areen09
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
areen09
Model Tree
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
Input Modalities
Output Modalities
Supported Functionality
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 (30B Total / 3.5B Active parameters, Hybrid Mamba-2 + Transformer + MoE)nvidia/Nemotron-SFT-Math-v4tinker://39a8c183-c679-5164-aaa0-e3355eff3e96:train:0/sampler_weights/final-pilot-lora-samplerpython
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_id = "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16"adapter_id = "areen09/Nemotron-3.5-Lightning-30B-A3B-Math-LoRA" tokenizer = AutoTokenizer.from_pretrained(base_model_id)base_model = AutoModelForCausalLM.from_pretrained( base_model_id, torch_dtype=torch.bfloat16, device_map="auto") # Load the LoRA adaptermodel = PeftModel.from_pretrained(base_model, adapter_id) # Prompt with mathematical reasoningmessages = [ {"role": "user", "content": "Solve for x: 3x^2 - 12x + 9 = 0."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)outputs = model.generate(inputs, max_new_tokens=1024, temperature=0.6)print(tokenizer.decode(outputs[0], skip_special_tokens=False))bash
vllm serve nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 \ --enable-lora \ --lora-modules math-lora=areen09/Nemotron-3.5-Lightning-30B-A3B-Math-LoRAimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_id = "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16"adapter_id = "areen09/Nemotron-3.5-Lightning-30B-A3B-Math-LoRA" tokenizer = AutoTokenizer.from_pretrained(base_model_id)base_model = AutoModelForCausalLM.from_pretrained( base_model_id, torch_dtype=torch.bfloat16, device_map="auto") # Load the LoRA adaptermodel = PeftModel.from_pretrained(base_model, adapter_id) # Prompt with mathematical reasoningmessages = [ {"role": "user", "content": "Solve for x: 3x^2 - 12x + 9 = 0."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)outputs = model.generate(inputs, max_new_tokens=1024, temperature=0.6)print(tokenizer.decode(outputs[0], skip_special_tokens=False))