Health Food Classifier (LoRA / PEFT)
A lightweight LoRA fine-tuned TinyLlama model that classifies foods as either Healthy or Unhealthy.
This project demonstrates how to perform parameter-efficient fine-tuning (PEFT) using LoRA adapters on top of the base TinyLlama model.
Instead of training the full model weights, only small LoRA adapter layers were trained, making the model significantly smaller and more efficient to train.
Model Details
- Base Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
- Fine-Tuning Method: LoRA / PEFT
- Task: Text Generation / Classification
- Intended Use:
- Educational
- Learning PEFT/LoRA
- Small demo classification project
What Is LoRA?
LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique.
Instead of updating all model weights:
- The original TinyLlama weights remain frozen
- Small trainable adapter layers are added
- Only the adapter weights are trained
Benefits:
- Much smaller model size
- Faster training
- Lower VRAM usage
- Easier sharing and deployment
Healthy Foods Dataset
Dataset:
healthyfoods
The model was trained using this prompt format:
### Question:
Is Apples healthy or unhealthy?
### Answer:
Healthy
Another example:
### Question:
Is French Fries healthy or unhealthy?
### Answer:
Unhealthy
Example Python Usage
Install dependencies:
pip install transformers peft torch
Load the model:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
base_model = AutoModelForCausalLM.from_pretrained(base_model_name)
model = PeftModel.from_pretrained(
base_model,
"YOUR_USERNAME/health-food-demo-lora"
)
tokenizer = AutoTokenizer.from_pretrained(
"YOUR_USERNAME/health-food-demo-lora"
)
prompt = """
### Question:
Is Pizza healthy or unhealthy?
### Answer:
"""
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(
**inputs,
max_new_tokens=10
)
print(tokenizer.decode(output[0], skip_special_tokens=True))
Example Output
### Question:
Is Pizza healthy or unhealthy?
### Answer:
Unhealthy
Training
The model was trained using supervised fine-tuning with custom food classification examples.
Example dataset format:
### Question:
Is Salmon healthy or unhealthy?
### Answer:
Healthy
Training characteristics:
- Prompt tokens were masked during loss computation
- The model only learned to predict answer tokens
- LoRA adapters were trained instead of full model weights
LoRA Configuration
Example LoRA settings used:
LoraConfig(
task_type=TaskType.CAUSAL_LM,
r=16,
lora_alpha=32,
lora_dropout=0.05,
bias="none"
)
GGUF Support
This repository may also include a GGUF LoRA adapter for llama.cpp-compatible runtimes.
Compatible tools:
- llama.cpp
- LM Studio
- KoboldCpp
- Text Generation WebUI
Example llama.cpp usage:
llama-cli \
-m TinyLlama-1.1B-Chat-v1.0-Q4_K_M.gguf \
--lora health_food_demo_lora.gguf \
-p "### Question: Is Pizza healthy or unhealthy?\n\n### Answer:"
Files
Table with columns: File, Description| File | Description |
|---|
adapter_model.safetensors | LoRA adapter weights |
adapter_config.json | LoRA configuration |
tokenizer.json | Tokenizer |
README.md | Documentation |
Optional:
Table with columns: File, Description| File | Description |
|---|
health_food_demo_lora.gguf | GGUF LoRA adapter for llama.cpp |
Limitations
- Small educational demo model
- Very limited dataset
- May hallucinate or answer incorrectly
- Not suitable for medical or nutritional advice
License
MIT