GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
build-small-hackathon
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
build-small-hackathon
Model Tree
unsloth/NVIDIA-Nemotron-3-Nano-4B
Input Modalities
Output Modalities
Supported Functionality
build-small-hackathon/noir-verdict-nemotron-4b-lorasave_pretrained_merged(..., save_method="merged_16bit") (Unsloth)python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "build-small-hackathon/noir-verdict-nemotron-4b-merged"tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( repo, torch_dtype=torch.bfloat16, trust_remote_code=True,).cuda().eval()The chat template is the Nemotron 3 chat template, with
enable_thinking=False baked in. The system prompt for an active
interrogation is built by engine/prompts.py:build_system_prompt(...).
python
messages = [ {"role": "system", "content": "You are Greta Lindholm, junior continuity writer at WJBK. ..."}, {"role": "user", "content": "Where were you at the time of the theft?"},]text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)n_ctx ≥ 4096temperature 0.6–0.7, top_p 0.9–0.95max_new_tokens 180–280 per turn<|im_end|>nvidia/cuda:12.8.1-devel-ubuntu22.04 + Python 3.13model.save_pretrained_merged(..., save_method="merged_16bit") in the same Modal jobtrain/modal_finetune.pybuild-small-hackathon/noir-verdict-nemotron-4b-lora (40.5 MB)build-small-hackathon/noir-verdict-nemotron-4b-gguf (2.84 GB)Apache-2.0. The base Nemotron 3 Nano weights are governed by NVIDIA's model license; the merged checkpoint and training code in this repo are Apache-2.0.
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "build-small-hackathon/noir-verdict-nemotron-4b-merged"tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( repo, torch_dtype=torch.bfloat16, trust_remote_code=True,).cuda().eval()messages = [ {"role": "system", "content": "You are Greta Lindholm, junior continuity writer at WJBK. ..."}, {"role": "user", "content": "Where were you at the time of the theft?"},]text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)