d1wash

mistral-7b-healthcare-qlora

Deploy Dedicated

README

License: apache-2.0

Model Details

Model Description

Developed by: Diwash Adhikari (d1wash)
Model type: Causal language model, LoRA adapter (PEFT)
Language(s): English
License: Apache 2.0
Finetuned from model: mistralai/Mistral-7B-Instruct-v0.2

Model Sources

Base model: mistralai/Mistral-7B-Instruct-v0.2
Training dataset: ChatDoctor-HealthCareMagic-100k
GitHub: Diwash17

Uses

Direct Use

This adapter is intended to be loaded on top of the Mistral-7B-Instruct-v0.2 base model to generate conversational responses in a medical Q&A style, for research, demonstration, and educational purposes.

Out-of-Scope Use

Not for clinical or diagnostic use. This model has not been validated for real-world medical decision-making and should never be used to diagnose, treat, or advise on actual patient care.
Not intended for use in any high-stakes setting without human expert review.

Bias, Risks, and Limitations

The model was fine-tuned on a public, community-sourced medical Q&A dataset (ChatDoctor-HealthCareMagic), which may contain inaccuracies, inconsistent medical advice, or outdated information.
Like all LLMs, the model can hallucinate plausible-sounding but incorrect medical claims.
Training was done on a subset of the full dataset (15,000 of ~110,000 examples) under a fixed compute/time budget, so coverage of rarer conditions or edge cases is limited.
No clinical validation, bias auditing, or safety red-teaming has been performed on this model.

Recommendations

Users should treat all outputs as illustrative only, verify any medical claims with a qualified professional, and not rely on this model for real health decisions.

How to Get Started with the Model

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_name = "mistralai/Mistral-7B-Instruct-v0.2"
adapter_repo = "d1wash/mistral-7b-healthcare-qlora"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
tokenizer.pad_token = tokenizer.eos_token

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=bnb_config,
    device_map="auto",
)

model = PeftModel.from_pretrained(base_model, adapter_repo)

prompt = """<s>[INST] You are a helpful medical assistant.
I have been experiencing persistent headaches for the past week. What could be the cause?

Patient: I wake up with a throbbing headache every morning. [/INST]

Doctor:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=200,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.3,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Training Data

A filtered, subsampled subset of ChatDoctor-HealthCareMagic-100k:

Original dataset size: ~112,000 examples
After quality filtering (removing very short/long examples): ~109,800 examples
Subsampled to 15,000 training examples and 1,500 evaluation examples to fit within a fixed compute/time budget on a single T4 GPU

Training Procedure

Preprocessing

Prompts formatted in Mistral's instruction format: <s>[INST] ... [/INST] ...
Tokenized with a max sequence length of 256 tokens (truncated/padded)

Training Hyperparameters

Quantization: 4-bit NF4 (bitsandbytes), double quantization enabled
Compute dtype: float16
LoRA rank (r): 8
LoRA alpha: 16
LoRA dropout: 0.05
Target modules: q_proj, v_proj
Optimizer: paged_adamw_8bit
Learning rate: 2e-4 (cosine schedule)
Effective batch size: 16 (per-device batch size 8 × gradient accumulation 2)
Total steps: 900 (full run completed — checkpoints saved through step 900)
Training regime: fp16 mixed precision, gradient checkpointing enabled

Speeds, Sizes, Times

Hardware: 1x NVIDIA T4 (16GB), Lightning AI Studio (free tier)
Training time: ~3 hours for the full 900-step run
Adapter size: ~13.6 MB (LoRA adapter only, does not include base model weights)

Evaluation

Metrics

Training and validation loss decreased consistently across checkpoints:

Table with columns: Step, Training Loss, Validation Loss
Step	Training Loss	Validation Loss
150	2.004	2.011
300	1.900	1.907
450	1.880	1.885
600	1.903	1.870

Validation loss tracked closely with training loss throughout, indicating the model generalized to the held-out set rather than memorizing the training data. Training completed the full 900-step run with the best checkpoint (by evaluation loss) restored at the end.

Technical Specifications

Compute Infrastructure

Hardware

NVIDIA T4 GPU (16GB VRAM), 4 vCPUs — Lightning AI Studio free tier

Software

transformers, peft, bitsandbytes, accelerate, trl
Python 3.12

Model Card Contact

Diwash Adhikari — d1wash on Hugging Face | GitHub: Diwash17

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

d1wash

Model Tree

Base

mistralai/Mistral-7B-Instruct-v0.2

Adapter

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Container

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Details

Model Description

Developed by: Diwash Adhikari (d1wash)
Model type: Causal language model, LoRA adapter (PEFT)
Language(s): English
License: Apache 2.0
Finetuned from model: mistralai/Mistral-7B-Instruct-v0.2

Model Sources

Base model: mistralai/Mistral-7B-Instruct-v0.2
Training dataset: ChatDoctor-HealthCareMagic-100k
GitHub: Diwash17

Uses

Direct Use

Out-of-Scope Use

Not for clinical or diagnostic use. This model has not been validated for real-world medical decision-making and should never be used to diagnose, treat, or advise on actual patient care.
Not intended for use in any high-stakes setting without human expert review.

Bias, Risks, and Limitations

The model was fine-tuned on a public, community-sourced medical Q&A dataset (ChatDoctor-HealthCareMagic), which may contain inaccuracies, inconsistent medical advice, or outdated information.
Like all LLMs, the model can hallucinate plausible-sounding but incorrect medical claims.
Training was done on a subset of the full dataset (15,000 of ~110,000 examples) under a fixed compute/time budget, so coverage of rarer conditions or edge cases is limited.
No clinical validation, bias auditing, or safety red-teaming has been performed on this model.

Recommendations

Users should treat all outputs as illustrative only, verify any medical claims with a qualified professional, and not rely on this model for real health decisions.

How to Get Started with the Model

python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_name = "mistralai/Mistral-7B-Instruct-v0.2"
adapter_repo = "d1wash/mistral-7b-healthcare-qlora"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
tokenizer.pad_token = tokenizer.eos_token

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=bnb_config,
    device_map="auto",
)

model = PeftModel.from_pretrained(base_model, adapter_repo)

prompt = """<s>[INST] You are a helpful medical assistant.
I have been experiencing persistent headaches for the past week. What could be the cause?

Patient: I wake up with a throbbing headache every morning. [/INST]

Doctor:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=200,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.3,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Training Data

A filtered, subsampled subset of ChatDoctor-HealthCareMagic-100k:

Original dataset size: ~112,000 examples
After quality filtering (removing very short/long examples): ~109,800 examples
Subsampled to 15,000 training examples and 1,500 evaluation examples to fit within a fixed compute/time budget on a single T4 GPU

Training Procedure

Preprocessing

Prompts formatted in Mistral's instruction format: <s>[INST] ... [/INST] ...
Tokenized with a max sequence length of 256 tokens (truncated/padded)

Training Hyperparameters

Quantization: 4-bit NF4 (bitsandbytes), double quantization enabled
Compute dtype: float16
LoRA rank (r): 8
LoRA alpha: 16
LoRA dropout: 0.05
Target modules: q_proj, v_proj
Optimizer: paged_adamw_8bit
Learning rate: 2e-4 (cosine schedule)
Effective batch size: 16 (per-device batch size 8 × gradient accumulation 2)
Total steps: 900 (full run completed — checkpoints saved through step 900)
Training regime: fp16 mixed precision, gradient checkpointing enabled

Speeds, Sizes, Times

Hardware: 1x NVIDIA T4 (16GB), Lightning AI Studio (free tier)
Training time: ~3 hours for the full 900-step run
Adapter size: ~13.6 MB (LoRA adapter only, does not include base model weights)

Evaluation

Metrics

Training and validation loss decreased consistently across checkpoints:

Table with columns: Step, Training Loss, Validation Loss
Step	Training Loss	Validation Loss
150	2.004	2.011
300	1.900	1.907
450	1.880	1.885
600	1.903	1.870