Pipeline

markdown
Copy code
Qwen/Qwen3-0.6B (base)
↓ SFT with LoRA
qwen3-0.6b-sft-dolly (this model)
↓ DPO
qwen3-0.6b-dpo-ultrafeedback

LoRA Configuration

Dataset

• Name: databricks/databricks-dolly-15k
• Subset: 3,000 samples (seed=42)
• Format: Instruction + Context → Response

Results

How to Use

python
Copy code
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base + adapter
tokenizer = AutoTokenizer.from_pretrained(
    "Shaheer05/qwen3-0.6b-sft-dolly",
    trust_remote_code=True
)
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-0.6B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)
model = PeftModel.from_pretrained(
    base,
    "Shaheer05/qwen3-0.6b-sft-dolly"
)

# Inference
messages = [{"role": "user", "content": "What is machine learning?"}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Platform

• Google Colab (Free Tier) — NVIDIA T4 GPU