GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
m4xx101
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
m4xx101
Model Tree
unsloth/Llama-3.2-3B-Instruct-bnb-4bit
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "m4xx101/llama-3.2-3b-lora"base_model = "unsloth/Llama-3.2-3B-Instruct-bnb-4bit" model = AutoModelForCausalLM.from_pretrained( base_model, load_in_4bit=True, torch_dtype=torch.bfloat16, device_map="auto",)model = PeftModel.from_pretrained(model, model_id)tokenizer = AutoTokenizer.from_pretrained(model_id) messages = [ {"role": "system", "content": "You are a helpful AI assistant."}, {"role": "user", "content": "Explain quantum computing in one paragraph."},]inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)print(tokenizer.decode(outputs[0], skip_special_tokens=True))python
merged = model.merge_and_unload()merged.save_pretrained("./llama-3.2-3b-openhermes-merged")tokenizer.save_pretrained("./llama-3.2-3b-openhermes-merged")| Parameter | Value |
|---|---|
| Base model | unsloth/Llama-3.2-3B-Instruct (4-bit BnB) |
| Dataset | teknium/OpenHermes-2.5 |
| Method | QLoRA (4-bit NF4 + LoRA) |
| LoRA rank (r) | 16 |
| LoRA alpha | 16 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Training steps | 30 |
| Final loss | 0.72 |
| Framework | Unsloth + PEFT 0.18.1 |
Apache 2.0 — same as the base Llama 3.2 model. Use it, merge it, stack it, ship it.
Built with Unsloth — the fastest way to fine-tune open LLMs on consumer hardware.
import torchfrom peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer model_id = "m4xx101/llama-3.2-3b-lora"base_model = "unsloth/Llama-3.2-3B-Instruct-bnb-4bit" model = AutoModelForCausalLM.from_pretrained( base_model, load_in_4bit=True, torch_dtype=torch.bfloat16, device_map="auto",)model = PeftModel.from_pretrained(model, model_id)tokenizer = AutoTokenizer.from_pretrained(model_id) messages = [ {"role": "system", "content": "You are a helpful AI assistant."}, {"role": "user", "content": "Explain quantum computing in one paragraph."},]inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)print(tokenizer.decode(outputs[0], skip_special_tokens=True))merged = model.merge_and_unload()merged.save_pretrained("./llama-3.2-3b-openhermes-merged")tokenizer.save_pretrained("./llama-3.2-3b-openhermes-merged")