Mukilan06

MedGemma-Chromosome-VLM

README

License: apache-2.0

📋 Overview

MedGemma-Chromosome-VLM is a domain-adapted vision-language model for automated chromosome spread interpretation. Given a chromosome spread image, it generates an image-grounded structured report covering chromosome count, morphology, overlap status, image quality, findings, interpretation, and explicit uncertainty statements.

This repository ships LoRA adapter weights only, fine-tuned on top of Google's MedGemma-4B.

⚠️ Research use only. Not validated for clinical diagnosis, treatment decisions, prenatal diagnosis, or genetic counseling. All outputs require review by a qualified professional.

🔍 Model Details

Table with columns: Field, Value
Field	Value
Developed by	Mukilan A.M.
Base model	`unsloth/medgemma-4b-it-unsloth-bnb-4bit`
Model type	Vision-Language Model (VLM)
Architecture	MedGemma-4B + LoRA
Fine-tuning method	Parameter-Efficient Fine-Tuning (LoRA, rank 32)
Frameworks	Transformers · PEFT · TRL · Unsloth
Language	English

The model is fine-tuned with LoRA using completion-only supervised fine-tuning (SFT) — only assistant response tokens contribute to the loss. The visual encoder is frozen; only language projection layers are updated. The training objective favors conservative, image-grounded reports that avoid hallucinating abnormalities.

🎯 Intended Uses

✅ Direct Use

Chromosome spread interpretation
Automated cytogenetics report generation
Educational demonstrations
Biomedical AI research
Medical VLM benchmarking

Input: chromosome spread image Output: structured report — count, morphology, overlap, image quality, findings, impression, uncertainty

❌ Out-of-Scope

Clinical diagnosis
Patient treatment decisions
Disease confirmation
Prenatal diagnosis
Genetic counseling

Generated reports are for research purposes only.

Downstream applications: digital pathology workflows, laboratory decision-support systems, chromosome visualization tools, medical AI assistants, biomedical VLM research.

🏋️ Training Details

Dataset

Table with columns: Split, Images
Split	Images
Train	~4,200
Validation	300
Test	300

Each sample pairs a chromosome spread image (resized to 448 × 448) with a structured annotation: cytogenetic description, morphology, chromosome count, findings, interpretation, and uncertainty statement.

Image preprocessing: EXIF correction → RGB conversion → CLAHE enhancement → auto contrast → bicubic resizing → white letterbox padding.

LoRA Configuration

Table with columns: Parameter, Value
Parameter	Value
Rank (r)	32
Alpha	64
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
Vision encoder	Frozen
Language layers	Trainable

Hyperparameters

Table with columns: Parameter, Value, Parameter, Value
Parameter	Value	Parameter	Value
Epochs	6	Warmup ratio	0.05
Learning rate	8e-5	Max sequence length	2048
Optimizer	`paged_adamw_8bit`	Image size	448

Quantization

4-bit NF4 · Double Quantization · BF16 compute · Flash Attention 2 (when available)

📊 Evaluation

Model selection was performed on validation loss.

Table with columns: Metric, Value
Metric	Value
Best checkpoint	`checkpoint-250`
Validation loss	≈ 0.430

Additional evaluation axes: BLEU, ROUGE-L, token-level F1, and medical consistency evaluation.

⚙️ Hardware & Environmental Impact

Table

GPU	NVIDIA L4
Precision	BF16
Quantization	4-bit
Training time	~2.5–3 hours

⚠️ Biases & Limitations

Inherits limitations from base MedGemma and the fine-tuning dataset
Dataset skews toward normal chromosome count (46), limiting rare-abnormality generalization
Cannot replace expert cytogenetic interpretation
Possible degradation on low-quality input images
Conservative reporting style may omit subtle abnormalities

Ethical considerations: intended solely for research and education. Must not be used as an autonomous diagnostic system — all outputs require review by qualified medical professionals.

💻 Usage

bash
pip install unsloth peft transformers

python
from unsloth import FastVisionModel
from peft import PeftModel

base_model = FastVisionModel.from_pretrained(
    "unsloth/medgemma-4b-it-unsloth-bnb-4bit"
)

model = PeftModel.from_pretrained(
    base_model,
    "Mukilan06/MedGemma-Chromosome-VLM"
)

📖 Citation

If you use this model in academic work, please cite:

bibtex
@misc{mukilan2026medgemmachromosomevlm,
  title={MedGemma-Chromosome-VLM: A Parameter-Efficient Vision Language Model for Image-Grounded Cytogenetic Report Generation},
  author={Mukilan A. M.},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/Mukilan06/MedGemma-Chromosome-VLM}}
}

🙏 Acknowledgements

This work builds upon Google MedGemma, Gemma, Unsloth, Hugging Face Transformers, PEFT, and TRL. Special thanks to the open-source AI community for enabling efficient medical vision-language model development.

Made with 🧬 by Mukilan A.M.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.