Arpan-Das

tinyllama-alpaca-finetuned

README

License: apache-2.0

Model Details

Table with columns: Property, Details
Property	Details
Base Model	TinyLlama/TinyLlama-1.1B-Chat-v1.0
Model Type	Causal Language Model
Fine-tuning Method	QLoRA (4-bit quantization + LoRA)
Dataset	Stanford Alpaca (10,000 examples)
Training Hardware	Kaggle T4 GPU
Training Time	~90 minutes
LoRA Rank (r)	16
LoRA Alpha	32
Target Modules	q_proj, v_proj
Trainable Parameters	~5.2M (0.19% of total)

Training Results

Table with columns: Step, Training Loss, Validation Loss
Step	Training Loss	Validation Loss
100	0.9415	0.9436
200	0.9322	0.9387
300	0.9646	0.9337
400	0.9552	0.9301
500	0.9627	0.9291

No overfitting observed — training and validation loss remained closely aligned throughout.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

# Load base model with 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True
)

base_model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    quantization_config=bnb_config,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
)
tokenizer.pad_token = tokenizer.eos_token

# Load fine-tuned adapter
model = PeftModel.from_pretrained(
    base_model,
    "your-username/tinyllama-alpaca-finetuned"
)

# Run inference
prompt = """Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Explain what machine learning is in simple terms.

### Response:"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example Outputs

Instruction: Explain what machine learning is in simple terms.

Response:

Machine learning is a field that deals with learning patterns in large datasets. It involves the use of algorithms, statistical models, and data mining to identify patterns and relationships in complex data sets.

Instruction: Write a short poem about the ocean.

Response:

I stand at the seashore, an endless expanse of waves, Their sound a symphony, a melody in the air, The tides, ebb, and flow, a constant rhythm, Swirls of sand, seashells, and the shingle's surface...

Technical Details

What is QLoRA?

QLoRA combines two techniques to make fine-tuning large models accessible on consumer hardware:

4-bit Quantization — reduces model memory from ~4GB to ~0.5GB
LoRA (Low-Rank Adaptation) — trains only 0.19% of parameters via small adapter matrices

Why TinyLlama?

TinyLlama-1.1B offers a great balance between capability and efficiency, making it ideal for fine-tuning experiments on free-tier GPUs.

Training Setup

python
# Key dependencies
trl==0.8.6
transformers==4.40.0
peft==0.10.0
bitsandbytes>=0.39.0
accelerate>=0.20.0

Author

Arpan | AI/ML Learning Journey

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

Arpan-Das

Model Tree

Base

TinyLlama/TinyLlama-1.1B-Chat-v1.0

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Details

Table with columns: Property, Details
Property	Details
Base Model	TinyLlama/TinyLlama-1.1B-Chat-v1.0
Model Type	Causal Language Model
Fine-tuning Method	QLoRA (4-bit quantization + LoRA)
Dataset	Stanford Alpaca (10,000 examples)
Training Hardware	Kaggle T4 GPU
Training Time	~90 minutes
LoRA Rank (r)	16
LoRA Alpha	32
Target Modules	q_proj, v_proj
Trainable Parameters	~5.2M (0.19% of total)

Training Results

Table with columns: Step, Training Loss, Validation Loss
Step	Training Loss	Validation Loss
100	0.9415	0.9436
200	0.9322	0.9387
300	0.9646	0.9337
400	0.9552	0.9301
500	0.9627	0.9291

No overfitting observed — training and validation loss remained closely aligned throughout.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

# Load base model with 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True
)

base_model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    quantization_config=bnb_config,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
)
tokenizer.pad_token = tokenizer.eos_token

# Load fine-tuned adapter
model = PeftModel.from_pretrained(
    base_model,
    "your-username/tinyllama-alpaca-finetuned"
)

# Run inference
prompt = """Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Explain what machine learning is in simple terms.

### Response:"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example Outputs

Instruction: Explain what machine learning is in simple terms.

Response:

Machine learning is a field that deals with learning patterns in large datasets. It involves the use of algorithms, statistical models, and data mining to identify patterns and relationships in complex data sets.

Instruction: Write a short poem about the ocean.

Response:

I stand at the seashore, an endless expanse of waves, Their sound a symphony, a melody in the air, The tides, ebb, and flow, a constant rhythm, Swirls of sand, seashells, and the shingle's surface...

Technical Details

What is QLoRA?

QLoRA combines two techniques to make fine-tuning large models accessible on consumer hardware:

4-bit Quantization — reduces model memory from ~4GB to ~0.5GB
LoRA (Low-Rank Adaptation) — trains only 0.19% of parameters via small adapter matrices

Why TinyLlama?

TinyLlama-1.1B offers a great balance between capability and efficiency, making it ideal for fine-tuning experiments on free-tier GPUs.

Training Setup

python
# Key dependencies
trl==0.8.6
transformers==4.40.0
peft==0.10.0
bitsandbytes>=0.39.0
accelerate>=0.20.0

Author

Arpan | AI/ML Learning Journey