Model Overview
This model is a LoRA/QLoRA fine-tuned version of LLaMA-2-7B, specialized for answering questions related to health insurance.
It is designed to explain health insurance concepts in clear, plain English, similar to how a knowledgeable insurance representative or benefits counselor would explain them to a customer.
The model was fine-tuned on a synthetic instruction-following dataset containing health insurance questions and corresponding explanations.
Intended Use
The model is designed to answer questions such as:
- What is a deductible?
- What is coinsurance?
- How does an HMO differ from a PPO?
- What is an Explanation of Benefits (EOB)?
- How does Medicare work?
- What is the Affordable Care Act?
- What is a copayment?
- What happens when a claim is denied?
Example:
Instruction
What is a deductible in health insurance?
Response
A deductible is the amount you must pay out-of-pocket for covered healthcare services before your insurance company begins paying for eligible medical expenses. Once the deductible has been met, you generally share costs with the insurer according to your plan's copayment or coinsurance rules.
What the Model Can Do
The model is intended for:
- Health insurance question answering
- Health insurance education
- Explaining insurance terminology
- Explaining healthcare billing concepts
- Explaining claims processes
- Explaining Medicare and Medicaid basics
- Explaining Affordable Care Act (ACA) concepts
- Educational chatbot applications
- Customer support assistance for general insurance queries
What the Model is NOT Designed For
This model is not intended to:
- Predict insurance claim approvals
- Make underwriting decisions
- Process or file insurance claims
- Interpret individual insurance policy documents
- Provide personalized insurance recommendations
- Replace licensed insurance professionals, healthcare providers, or legal advisors
The model explains health insurance concepts in a general educational context and should not be considered medical, legal, or insurance advice.
Training Dataset
The model was fine-tuned using a synthetic instruction-following dataset focused on health insurance.
Each training sample follows the format:
### Instruction:
<Question>
### Response:
<Explanation>
Topics covered include:
- Deductibles
- Copayments
- Coinsurance
- Premiums
- Out-of-pocket maximums
- PPOs
- HMOs
- EPOs
- Medicare
- Medicaid
- Affordable Care Act (ACA)
- Claims and Appeals
- Explanation of Benefits (EOB)
- Health Savings Accounts (HSA)
- Flexible Spending Accounts (FSA)
- COBRA
- Provider Networks
- Medical Billing Terminology
Training Details
Base Model
Fine-tuning Method
- LoRA
- QLoRA
- Hugging Face Transformers
- PEFT
- TRL (SFTTrainer)
Hyperparameters
Table with columns: Parameter, Value| Parameter | Value |
|---|
| Epochs | 1 |
| Batch Size | 4 |
| Gradient Accumulation | 4 |
| Learning Rate | 2e-4 |
| Max Sequence Length | 512 |
| Optimizer | AdamW |
| Scheduler | Cosine |
| Quantization | 4-bit NF4 |
Evaluation
The model was evaluated using:
Results
ROUGE
Table with columns: Metric, Score| Metric | Score |
|---|
| ROUGE-1 | 0.5317 |
| ROUGE-2 | 0.2286 |
| ROUGE-L | 0.3614 |
BERTScore
Table with columns: Metric, Score| Metric | Score |
|---|
| Precision | 0.9153 |
| Recall | 0.9041 |
| F1 | 0.9096 |
Usage
from transformers import AutoTokenizer
from peft import AutoPeftModelForCausalLM
model_id = "Yoshita18/llama2-health-insurance"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoPeftModelForCausalLM.from_pretrained(
model_id,
device_map="auto"
)
prompt = """### Instruction:
Explain what coinsurance is.
### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=150,
do_sample=False,
)
print(
tokenizer.decode(
outputs[0][inputs["input_ids"].shape[1]:],
skip_special_tokens=True,
)
)
Limitations
This model has several limitations:
- Trained on a synthetic dataset.
- May generate inaccurate or outdated insurance information.
- Does not understand individual insurance policies.
- Cannot determine insurance claim eligibility or approval.
- Cannot replace professional medical, insurance, or legal advice.
- Responses should always be verified with official insurance providers, healthcare professionals, or policy documentation.
Intended Users
This model is suitable for:
- Insurance customers
- Healthcare consumers
- Students
- Researchers
- Developers building educational health insurance chatbots
- Organizations prototyping insurance assistance systems
License
This model is released under the Apache 2.0 License.
Base model:
Please ensure compliance with Meta's Llama 2 license agreement in addition to this repository's license when using this adapter.
Citation
If you use this model in your work, please cite this repository.
@misc{yoshita18_llama2_health_insurance,
title={LLaMA-2 Fine-Tuned for Health Insurance Question Answering},
author={Yoshita},
year={2026},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/Yoshita18/llama2-health-insurance}}
}