GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
vishnuamarapu
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
vishnuamarapu
Model Tree
Qwen/Qwen2.5-0.5B-Instruct
Input Modalities
Output Modalities
Supported Functionality
This repository contains a QLoRA (Quantized Low-Rank Adaptation) fine-tuned version of Qwen2.5-0.5B-Instruct.
The model was trained using QLoRA, where the base model was loaded in 4-bit precision during training to reduce GPU memory usage. Only the LoRA adapter weights were updated and are included in this repository.
Note: This repository contains only the QLoRA adapter. During inference, the adapter can be loaded with the original base model in FP32, FP16, BF16, 8-bit, or 4-bit depending on your hardware.
Qwen/Qwen2.5-0.5B-Instructbash
pip install transformers peft accelerate torch safetensorspython
import torchfrom transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"ADAPTER = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, torch_dtype=torch.float32) model = PeftModel.from_pretrained( base_model, ADAPTER) model.eval()python
messages = [ { "role": "system", "content": ( "You are Vishnu's personal AI assistant. " "Answer questions about Vishnu." ) }, { "role": "user", "content": "Tell me about Vishnu." }] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer( text, return_tensors="pt") with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=256, temperature=0.7, do_sample=True, top_p=0.9, ) response = tokenizer.decode( outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True,) print(response)python
import torchimport gradio as gr from transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"ADAPTER = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, torch_dtype=torch.float32) model = PeftModel.from_pretrained( base_model, ADAPTER) model.eval() SYSTEM_PROMPT = ( "You are Vishnu's personal AI assistant. " "Answer questions about Vishnu.") def predict(message, history): messages = [ { "role": "system", "content": SYSTEM_PROMPT, } ] for user, assistant in history: messages.append({"role": "user", "content": user}) messages.append({"role": "assistant", "content": assistant}) messages.append({"role": "user", "content": message}) text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, ) inputs = tokenizer( text, return_tensors="pt", ) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=256, temperature=0.7, do_sample=True, top_p=0.9, ) response = tokenizer.decode( outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True, ) return response gr.ChatInterface( fn=predict, title="Vishnu Personal AI (QLoRA)", description="Qwen2.5-0.5B fine-tuned using QLoRA",).launch()| Parameter | Value |
|---|---|
| max_new_tokens | 256 |
| temperature | 0.7 |
| top_p | 0.9 |
| torch_dtype | float32 (CPU Example) |
text
adapter_config.jsonadapter_model.safetensorschat_template.jinjatokenizer.jsontokenizer_config.jsonREADME.mdThis model was fine-tuned using QLoRA (Quantized Low-Rank Adaptation).
During training:
The uploaded repository contains only the adapter weights. During inference, these adapters are applied to the original Qwen2.5-0.5B-Instruct base model.
If you use this model in your work, please cite this repository.
bibtex
@misc{vishnu_qwen25_qlora, author = {Vishnu Amarapu}, title = {QLoRA Fine-Tuned Qwen2.5-0.5B-Instruct}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora}}}pip install transformers peft accelerate torch safetensorsimport torchfrom transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"ADAPTER = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, torch_dtype=torch.float32) model = PeftModel.from_pretrained( base_model, ADAPTER) model.eval()messages = [ { "role": "system", "content": ( "You are Vishnu's personal AI assistant. " "Answer questions about Vishnu." ) }, { "role": "user", "content": "Tell me about Vishnu." }] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer( text, return_tensors="pt") with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=256, temperature=0.7, do_sample=True, top_p=0.9, ) response = tokenizer.decode( outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True,) print(response)import torchimport gradio as gr from transformers import AutoTokenizer, AutoModelForCausalLMfrom peft import PeftModel BASE_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"ADAPTER = "vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL, torch_dtype=torch.float32) model = PeftModel.from_pretrained( base_model, ADAPTER) model.eval() SYSTEM_PROMPT = ( "You are Vishnu's personal AI assistant. " "Answer questions about Vishnu.") def predict(message, history): messages = [ { "role": "system", "content": SYSTEM_PROMPT, } ] for user, assistant in history: messages.append({"role": "user", "content": user}) messages.append({"role": "assistant", "content": assistant}) messages.append({"role": "user", "content": message}) text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, ) inputs = tokenizer( text, return_tensors="pt", ) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=256, temperature=0.7, do_sample=True, top_p=0.9, ) response = tokenizer.decode( outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True, ) return response gr.ChatInterface( fn=predict, title="Vishnu Personal AI (QLoRA)", description="Qwen2.5-0.5B fine-tuned using QLoRA",).launch()adapter_config.jsonadapter_model.safetensorschat_template.jinjatokenizer.jsontokenizer_config.jsonREADME.md@misc{vishnu_qwen25_qlora, author = {Vishnu Amarapu}, title = {QLoRA Fine-Tuned Qwen2.5-0.5B-Instruct}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/vishnuamarapu/Full-Fine-Tuning-Qwen-2.5-0.5B-instruct-QLora}}}