MindBridge-Llama
MindBridge-Llama is a LoRA-adapted conversational language model fine-tuned for supportive mental health dialogue. Built on top of khazarai/MentalChat-16K, it is designed to generate empathetic, supportive, and context-aware responses to mental health-related conversations.
Note: This repository contains only the LoRA adapter weights. The base model (khazarai/MentalChat-16K) must be downloaded separately.
Model Details
- Model Name: MindBridge-Llama
- Base Model: khazarai/MentalChat-16K
- Architecture: LLaMA + LoRA (PEFT)
- Framework: Hugging Face Transformers + PEFT
- Fine-tuned By: Smukherjee
Intended Use
This model is intended for:
- Mental health conversational AI
- Supportive dialogue generation
- Educational NLP projects
- Mental health LLM research
- Experimental chatbot development
This model is intended only for research, educational, and experimental purposes.
Training
MindBridge-Llama was fine-tuned using Low-Rank Adaptation (LoRA) on a curated collection of publicly available mental health conversational datasets.
The fine-tuning process included:
- Tokenization using the MentalChat tokenizer
- LoRA-based parameter-efficient fine-tuning with PEFT
- Supervised training using Hugging Face Transformers
- Evaluation on a held-out validation split
Compared to the base model, MindBridge-Llama is specialized for the MindBridge project and is intended to work alongside MindBridge-RoBERTa to provide more context-aware conversational responses.
Requirements
Install the required libraries before loading the model.
pip uninstall -y torchao
pip install peft transformers accelerate sentencepiece bitsandbytes
Note: If using Kaggle, restart the notebook session after uninstalling torchao.
Usage
Load the base model first and then apply the LoRA adapter.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
model_name = "khazarai/MentalChat-16K"
tokenizer = AutoTokenizer.from_pretrained(model_name)
base_model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map="auto"
)
model = PeftModel.from_pretrained(
base_model,
"S-mukherjee/MindBridge-Llama"
)
Inference Example
Use the tokenizer's chat template before generating a response.
messages = [
{
"role": "user",
"content": "I feel like nobody understands me anymore."
}
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
return_dict=True,
return_tensors="pt",
add_generation_prompt=True,
)
inputs = {k: v.to(model.device) for k, v in inputs.items()}
outputs = model.generate(
**inputs,
max_new_tokens=150,
temperature=0.7,
top_p=0.9,
do_sample=True,
repetition_penalty=1.1,
pad_token_id=tokenizer.eos_token_id,
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
Example
User
I feel like nobody understands me anymore.
Assistant
It sounds really tough and lonely to you. I'm sorry you're feeling this way. I'm here to listen if you'd like to talk about what's been making you feel this way.
Limitations
This model is intended only for research, educational, and experimental applications.
It is not a medical device and should not be used to diagnose mental health conditions or replace qualified mental health professionals. Generated responses may occasionally be inaccurate, incomplete, or inappropriate for sensitive situations. Human oversight is recommended for any real-world deployment.
Citation
If you use this model in research or projects, please cite this repository.
Acknowledgements
This model is built upon khazarai/MentalChat-16K and fine-tuned using the Hugging Face Transformers and PEFT libraries.
This conversational model is part of the MindBridge project.
For mental health text classification, please also check out:
🧠 MindBridge-RoBERTa
https://huggingface.co/S-mukherjee/MindBridge-RoBERTa
Together, these models form a two-stage AI-assisted mental health support pipeline:
- MindBridge-RoBERTa analyzes user text and identifies the mental health category.
- MindBridge-Llama generates an empathetic and context-aware conversational response based on the user's input and the identified mental health context.
Author
Smukherjee
Developed as part of the MindBridge project, which explores the use of Large Language Models and NLP for mental health support and conversational AI.