Model Description
PancCADx is an interpretable multimodal framework for pancreatic cancer diagnosis via endoscopic ultrasound (EUS). This adapter was trained using a two-stage alignment strategy:
- SFT (Supervised Fine-Tuning): Learning diagnostic patterns from expert annotations
- DPO (Direct Preference Optimization): Error-driven alignment to reduce hallucinations
Usage
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
base_model = Qwen3VLForConditionalGeneration.from_pretrained(
"Qwen/Qwen3-VL-8B-Thinking",
torch_dtype="auto",
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "shan1984/PancCADx-DPO")
processor = AutoProcessor.from_pretrained("shan1984/PancCADx-DPO")
Training Details
- Base model: Qwen3-VL-8B-Thinking
- LoRA config: rank=128, alpha=256, target=all
- DPO: beta=0.3, lr=1e-7, epochs=10
- Training framework: LLaMA-Factory
Table with columns: Metric, Value| Metric | Value |
|---|
| Sensitivity | 95.74% |
| Specificity | 77.78% |
| Accuracy | 89.53% |
Citation
@inproceedings{hu2026panccadx,
title={PancCADx: A Multimodal Framework for Pancreatic Cancer Diagnosis},
author={Hu, Shan and Xiao, Changhong and Qin, Xianzheng and Mei, Bin and Cheng, Bin and Wang, Zhongyuan},
booktitle={MICCAI},
year={2026}
}
License
Apache 2.0