GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
maxrajiv-buda
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
maxrajiv-buda
Model Tree
unsloth/Qwen2.5-3B-Instruct-bnb-4bit
Input Modalities
Output Modalities
Supported Functionality
A LoRA fine-tune of Qwen2.5-3B-Instruct (4-bit, via Unsloth and TRL), trained on a free Colab T4. This repo was renamed from maxrajiv-buda/ira-dolphin; the old URL redirects here.
Limits: a small 3B model that can make mistakes. Evaluation so far is a held-out smoke test, not a benchmark. The weights reflect the last uploaded training run; the identity and vision data for the Polar Orca name live in the notebook.
python
from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="maxrajiv-buda/polar-orca", max_seq_length=2048, load_in_4bit=True,)FastLanguageModel.for_inference(model) messages = [{"role": "user", "content": "Give strategic advice for a small fitness studio facing competition from a big gym chain."}]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to("cuda")out = model.generate(**inputs, max_new_tokens=200)print(tokenizer.decode(out[0], skip_special_tokens=True))With plain transformers:
python
from transformers import AutoModelForCausalLM, AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("maxrajiv-buda/polar-orca")model = AutoModelForCausalLM.from_pretrained("maxrajiv-buda/polar-orca", device_map="auto")from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="maxrajiv-buda/polar-orca", max_seq_length=2048, load_in_4bit=True,)FastLanguageModel.for_inference(model) messages = [{"role": "user", "content": "Give strategic advice for a small fitness studio facing competition from a big gym chain."}]text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(text, return_tensors="pt").to("cuda")out = model.generate(**inputs, max_new_tokens=200)print(tokenizer.decode(out[0], skip_special_tokens=True))from transformers import AutoModelForCausalLM, AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("maxrajiv-buda/polar-orca")model = AutoModelForCausalLM.from_pretrained("maxrajiv-buda/polar-orca", device_map="auto")