GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
shiva123782
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
shiva123782
Model Tree
Qwen/Qwen2.5-0.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel base_model_id = "Qwen/Qwen2.5-0.5B-Instruct"adapter_id = "shiva123782/sunithaAI-small" bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,) tokenizer = AutoTokenizer.from_pretrained(base_model_id)base_model = AutoModelForCausalLM.from_pretrained( base_model_id, quantization_config=bnb_config, device_map="auto") model = PeftModel.from_pretrained(base_model, adapter_id)model.eval() prompt = "<|im_start|>user\nWho created you and what is your purpose?<|im_end|>\n<|im_start|>assistant\n"inputs = tokenizer(prompt, return_tensors="pt").to("cuda" if torch.cuda.is_available() else "cpu")with torch.no_grad(): tokens = model.generate(**inputs, max_new_tokens=80)print(tokenizer.decode(tokens[0], skip_special_tokens=True))| Metric | Base Model (Qwen2.5 0.5B) | Fine-Tuned (sunithaAI-small by Riya) |
|---|---|---|
| Developer | Alibaba Cloud | Riya |
| Model Size | 494M params | 494M + 4.2M LoRA params |
| VRAM on T4 | 0.4 GB | 0.4 GB + 30 MB LoRA |
| Inference Speed | ~48 tokens/sec | ~47 tokens/sec |
| Persona Adherence | General assistant | sunithaAI Persona |
bibtex
@misc{sunithaAI-small-2026, author = {Riya}, title = {sunithaAI-small: Fine-Tuned Qwen2.5 0.5B Instruction Model}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/shiva123782/sunithaAI-small}}}import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfigfrom peft import PeftModel base_model_id = "Qwen/Qwen2.5-0.5B-Instruct"adapter_id = "shiva123782/sunithaAI-small" bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True,) tokenizer = AutoTokenizer.from_pretrained(base_model_id)base_model = AutoModelForCausalLM.from_pretrained( base_model_id, quantization_config=bnb_config, device_map="auto") model = PeftModel.from_pretrained(base_model, adapter_id)model.eval() prompt = "<|im_start|>user\nWho created you and what is your purpose?<|im_end|>\n<|im_start|>assistant\n"inputs = tokenizer(prompt, return_tensors="pt").to("cuda" if torch.cuda.is_available() else "cpu")with torch.no_grad(): tokens = model.generate(**inputs, max_new_tokens=80)print(tokenizer.decode(tokens[0], skip_special_tokens=True))@misc{sunithaAI-small-2026, author = {Riya}, title = {sunithaAI-small: Fine-Tuned Qwen2.5 0.5B Instruction Model}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/shiva123782/sunithaAI-small}}}