Model Description

This model takes an image of a skin lesion and a textual prompt as input, and outputs a structured JSON object containing key dermatological features. It was fine-tuned on a curated medical dataset of skin extracts to recognize and describe:

• Lesion Type: The clinical classification of the lesion (e.g., melanoma, nevus, basal cell carcinoma).
• Color: Predominant colors observed (e.g., brown, black, red, white, blue).
• Symmetry: Assessment of the lesion's symmetry (e.g., symmetric, asymmetric).
• Borders: Description of the lesion's edges (e.g., regular, irregular, scalloped).
• Texture: Surface characteristics (e.g., smooth, rough, scaly, ulcerated).
• Summary: A concise, professional clinical summary of the findings.

Training Details

• Base Model: Qwen/Qwen3-VL-2B-Instruct
• Training Framework: Unsloth (enabling up to 2x faster training and 50% less memory usage via optimized LoRA/QLoRA).
• Dataset: Skin_Extract
• Optimization: 4-bit quantization (QLoRA) with rank r=16, alpha 32, and dropout 0.05 (adjust these values to match your actual Unsloth config).

Evaluation Metrics

The model's extraction accuracy was evaluated using BERTScore to measure the semantic similarity between the model's generated JSON fields and the clinical ground truth annotations.

Below are the detailed BERTScore (F1) statistics for each extracted feature:

Note: The high mean scores (>0.89 across all fields) and low standard deviations indicate that the model consistently generates descriptions that are semantically highly aligned with the clinical ground truth, with minimal variance.

Limitations and Bias

• Not a Diagnostic Tool: The model can hallucinate or misclassify rare conditions. Human oversight is strictly required.
• Dataset Bias: Performance may vary depending on skin tones, image quality, lighting conditions, and the specific demographics represented in the training dataset.
• Scope: The model is strictly tuned for cutaneous/dermatological images and will perform poorly or unpredictably on non-skin medical imagery (e.g., X-rays, MRIs).

---

Developed by: 0xAgamy