Model Details
Table with columns: Property, Value| Property | Value |
|---|
| Base model | JibayAi/Jibay_2 (2B parameters) |
| Fine-tuning method | QLoRA (4-bit + LoRA adapters) |
| LoRA rank | 16 |
| LoRA alpha | 16 |
| LoRA dropout | 0 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Training framework | Unsloth + TRL SFTTrainer |
| Context length | 2048 tokens |
| License | Apache 2.0 |
| Language | English |
Intended Use
Primary Use
This model helps healthcare professionals quickly understand a patient's background by transforming fragmented clinical shorthand (e.g., pt 58 yo M c/o chest pain x2d hx HTN DM2) into a clean, structured summary organized into standardized sections:
- Patient Background
- Chief Complaint
- Medical History
- Medications
- Vital Signs
- Examination
- Investigations
- Assessment
Intended Users
- Clinical researchers
- Medical documentation teams
- Healthcare AI developers
- Medical education platforms
Out-of-Scope Uses
- Not a diagnostic tool. The model does not diagnose, treat, or prescribe.
- Not for direct clinical decision-making. All outputs must be reviewed by a qualified healthcare professional.
- Not for processing real patient data without de-identification. Using PHI without proper safeguards may violate HIPAA/GDPR.
- Not multilingual. Trained only on English clinical text.
Training Data
Data Sources
- TheBlueScrubs-v1-fixed (openmed-community) — filtered medical text corpus
- Hindawi medical journals (mkurman/hindawi-journals-2007-2023) — open-access medical literature
Data Pipeline
- Raw medical text was filtered to identify genuine clinical notes using a keyword-based heuristic (~2,645 from 70,000 total examples).
- Each clinical note was converted into messy shorthand using a rule-based transformation (medical abbreviations, structure removal, connective tissue stripping).
- Qwen2.5-7B-Instruct generated clean, structured summaries from the messy notes, producing a parallel corpus of
{messy, clean} pairs.
- Pairs passing quality filters were retained.
Dataset Statistics
Table with columns: Split, Examples| Split | Examples |
|---|
| Train | 2,107 |
| Validation | 263 |
| Test | 264 |
| Total | 2,634 |
- Average messy input length: ~900 characters
- Average clean output length: ~803 characters
Data Limitations
- Rule-based "messy" transformation may not perfectly reflect real-world clinical shorthand.
- Source corpus contains heterogeneous medical content (case reports, journal abstracts, patient notes).
- No formal medical validation of generated summaries was performed.
Training Procedure
Training Hyperparameters
Table with columns: Hyperparameter, Value| Hyperparameter | Value |
|---|
| Epochs | 3 |
| Per-device batch size | 16 |
| Gradient accumulation steps | 1 |
| Effective batch size | 16 |
| Learning rate | 2e-4 |
| LR scheduler | Cosine |
| Warmup steps | 20 |
| Weight decay | 0.01 |
| Max grad norm | 1.0 |
Training Framework
- Unsloth for 2x faster fine-tuning and reduced memory usage
- TRL SFTTrainer for supervised fine-tuning
- PEFT for LoRA adapters
- BitsAndBytes for 4-bit quantization
Hardware
- Google Colab Pro
- NVIDIA A100 (40 GB)
Evaluation Strategy
- Validation loss evaluated every 20 steps
- Early stopping with patience of 3 evaluations
- Best checkpoint loaded based on lowest validation loss
Training Results
- Best validation loss: 1.4403
- Training time: ~5.7 minutes
Usage
Loading the Model
from transformers import AutoModelForCausalLM, AutoTokenizerfrom peft import PeftModelimport torch BASE_MODEL = "JibayAi/Jibay_2"ADAPTER = "aradmanamnaoon/jibay2-medical-v2" tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)base = AutoModelForCausalLM.from_pretrained( BASE_MODEL, torch_dtype=torch.float16, device_map="auto",)model = PeftModel.from_pretrained(base, ADAPTER)model = model.merge_and_unload()
Running Inference
prompt = '### Input:\n45yo F abd pain 2d nausea no vomit.\n\n### Clean Summary:\n' inputs = tokenizer(prompt, return_tensors='pt').to(model.device)outputs = model.generate(**inputs, max_new_tokens=600, temperature=0.7)result = tokenizer.decode(outputs[0], skip_special_tokens=True)print(result.split('### Clean Summary:')[-1].strip())
## Patient Background- 45-year-old female## Chief Complaint- Abdominal pain for 2 days, associated with nausea without vomiting## Medical History- Appendectomy (2019)## Vital Signs- Temperature: 37.8C- Blood pressure: 120/80 mmHg## Examination- Abdomen: Soft, tender in the right lower quadrant
Limitations
Known Limitations
- Hallucination risk. The model may invent clinical details not present in the input. Always verify outputs against the source note.
- Not clinically validated. No medical professional has reviewed the model's outputs for accuracy.
- Training data bias. The model inherits biases from the underlying corpus.
- Limited to English. Does not support multilingual clinical notes.
- Rule-based messy transformation. Messy inputs during training were generated procedurally, not from real clinical shorthand.
- Small training set. 2,634 examples is small; performance may degrade on out-of-distribution inputs.
Ethical Considerations
- Human-in-the-loop required. This model is a documentation aid, not a replacement for clinical judgment.
- PHI handling. Do not feed protected health information without explicit consent and appropriate safeguards.
- No clinical deployment. Not cleared or approved by any regulatory body (FDA, EMA, MHRA, etc.).
Bias and Fairness
- Not evaluated for demographic fairness across age, sex, ethnicity, or socioeconomic groups.
- Performance may vary significantly across patient populations.
Environmental Impact
Training used a single NVIDIA A100 GPU for approximately 6 minutes.
Citation
@misc{jibay2-medical-v2, title = {Jibay2-Medical-v2: Fine-tuned Medical Note Summarizer}, author = {aradmanamnaoon}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/aradmanamnaoon/jibay2-medical-v2}}}
Acknowledgments
- JibayAi for the base Jibay 2 model
- Unsloth for the training framework
- Qwen team for Qwen2.5-7B-Instruct used in data generation
- Hugging Face for model hosting
Disclaimer: This model is for research purposes only. It is not a medical device and must not be used for clinical decision-making without review by a qualified healthcare professional.