zaid646

instruction-follower-lora

Model Details

The adapter was trained using peft + bitsandbytes with the following configuration:

Table with columns: Hyperparameter, Value
Hyperparameter	Value
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.1
Target modules	q_proj, v_proj, k_proj, o_proj
Quantization	4-bit NF4, double quant
Batch size	2 (effective 16 with grad accum)
Learning rate	1e-4
Epochs	2
Max sequence length	512
Warmup steps	50
Optimizer	AdamW (paged)

Training Results

Table with columns: Metric, Value
Metric	Value
Final loss	1.22
Train samples/sec	6.96
Train steps/sec	0.43

How to Use

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Base model
base_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter_name = "zaid646/tinyllama-1.1b-alpaca-qlora"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
tokenizer.pad_token = tokenizer.eos_token

# Load model with 4-bit quantization
quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
)

model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=quant_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

# Load adapter
model = PeftModel.from_pretrained(model, adapter_name)

# Inference
prompt = "### Instruction:\nExplain what machine learning is.\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Recipe

This model was trained using the fine-tuning-recipes framework. To reproduce:

bash
git clone https://github.com/ZAID646/fine-tuning-recipes.git
cd fine-tuning-recipes
pip install -e .
python -m src.cli train --config recipes/qlora.yaml

Limitations

Fine-tuned on English Alpaca data only — performance on other languages may vary
1.1B parameter base model — may not match larger models on complex reasoning
Instruction format must match Alpaca style (### Instruction:\n...\n### Response:\n)

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

zaid646

Model Tree

Base

TinyLlama/TinyLlama-1.1B-Chat-v1.0

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

Model Details

The adapter was trained using peft + bitsandbytes with the following configuration:

Table with columns: Hyperparameter, Value
Hyperparameter	Value
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.1
Target modules	q_proj, v_proj, k_proj, o_proj
Quantization	4-bit NF4, double quant
Batch size	2 (effective 16 with grad accum)
Learning rate	1e-4
Epochs	2
Max sequence length	512
Warmup steps	50
Optimizer	AdamW (paged)

Training Results

Table with columns: Metric, Value
Metric	Value
Final loss	1.22
Train samples/sec	6.96
Train steps/sec	0.43

How to Use

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Base model
base_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter_name = "zaid646/tinyllama-1.1b-alpaca-qlora"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
tokenizer.pad_token = tokenizer.eos_token

# Load model with 4-bit quantization
quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
)

model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=quant_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

# Load adapter
model = PeftModel.from_pretrained(model, adapter_name)

# Inference
prompt = "### Instruction:\nExplain what machine learning is.\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Recipe

This model was trained using the fine-tuning-recipes framework. To reproduce:

bash
git clone https://github.com/ZAID646/fine-tuning-recipes.git
cd fine-tuning-recipes
pip install -e .
python -m src.cli train --config recipes/qlora.yaml

Limitations

Fine-tuned on English Alpaca data only — performance on other languages may vary
1.1B parameter base model — may not match larger models on complex reasoning
Instruction format must match Alpaca style (### Instruction:\n...\n### Response:\n)

instruction-follower-lora

README

Model Details

Training Results

How to Use

Training Recipe

Limitations

Explore FriendliAI today

README

Model Details

Training Results

How to Use

Training Recipe

Limitations