rudrakshrakeshzodage

CustomerGPT-0.5B-LoRA

README

License: apache-2.0

📊 Dataset Specifications

The pipeline processes a raw dataset containing 26,872 support requests and expands it via data augmentation to enhance robustness.

1. Data Splits & Volume

Raw Records: 26,872
Cleaned & Filtered Records: 26,872 (100% pass rate at Quality Threshold)
Augmented Records: 107,488 (80,616 synthetic additions)

Table with columns: Split, Percentage, Records Count, Purpose
Split	Percentage	Records Count	Purpose
Train	80%	85,990	Model gradient updates
Validation	10%	10,748	Early stopping & hyperparameter tuning
Test	10%	10,750	Final evaluation

⚙️ Model Training Specifications

Table with columns: Parameter, Configuration / Specification
Parameter	Configuration / Specification
Base Model	`Qwen/Qwen2.5-0.5B-Instruct`
Training Method	QLoRA (4-bit Quantization + LoRA)
Quantization Scheme	NormalFloat4 (`nf4`), Double Quantization, `float16` compute type
LoRA Rank ( $r$ )	16
LoRA Alpha ( $α$ )	32

📈 Fine-Tuning Metrics

Total Training Time: 2,293.04 seconds (~38.2 minutes)
Training Speed: 0.305 steps/second (~3.28 seconds/step)
Initial Loss: 2.4139
Final Training Loss: 0.6336
Average Training Loss: 0.8391

🚀 How to Load and Run Inference

You can load this LoRA adapter directly using the transformers and peft libraries:

python

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModel base_model_name = "Qwen/Qwen2.5-0.5B-Instruct"adapter_id = "rudrakshrakeshzodage/CustomerGPT-0.5B-LoRA" # Load Tokenizertokenizer = AutoTokenizer.from_pretrained(base_model_name) # Load Base Modelbase_model = AutoModelForCausalLM.from_pretrained(    base_model_name,    torch_dtype=torch.float16,    device_map="auto") # Attach LoRA Adaptermodel = PeftModel.from_pretrained(base_model, adapter_id)model.eval() # Chat Format Examplesystem_prompt = "You are CustomerGPT, a helpful and professional customer support assistant."messages = [    {"role": "system", "content": system_prompt},    {"role": "user", "content": "How to newsletter subscription"}] formatted_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = tokenizer(formatted_prompt, return_tensors="pt").to("cuda") with torch.no_grad():    outputs = model.generate(        **inputs,         max_new_tokens=256,        temperature=0.7,        top_p=0.9,        repetition_penalty=1.1,        pad_token_id=tokenizer.pad_token_id,        eos_token_id=tokenizer.eos_token_id    )    response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)print(response)

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

rudrakshrakeshzodage

Model Tree

Base

Qwen/Qwen2.5-0.5B-Instruct

Adapter

this model

Input Modalities

Text

Output Modalities