Model Description

PancCADx is an interpretable multimodal framework for pancreatic cancer diagnosis via endoscopic ultrasound (EUS). This adapter was trained using a two-stage alignment strategy:

1. SFT (Supervised Fine-Tuning): Learning diagnostic patterns from expert annotations
2. DPO (Direct Preference Optimization): Error-driven alignment to reduce hallucinations

Usage

python
Copy code
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from peft import PeftModel

# Load base model
base_model = Qwen3VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen3-VL-8B-Thinking",
    torch_dtype="auto",
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "shan1984/PancCADx-DPO")
processor = AutoProcessor.from_pretrained("shan1984/PancCADx-DPO")

Training Details

• Base model: Qwen3-VL-8B-Thinking
• LoRA config: rank=128, alpha=256, target=all
• DPO: beta=0.3, lr=1e-7, epochs=10
• Training framework: LLaMA-Factory

Performance (External Validation, n=191)

Citation

bibtex
Copy code
@inproceedings{hu2026panccadx,
  title={PancCADx: A Multimodal Framework for Pancreatic Cancer Diagnosis},
  author={Hu, Shan and Xiao, Changhong and Qin, Xianzheng and Mei, Bin and Cheng, Bin and Wang, Zhongyuan},
  booktitle={MICCAI},
  year={2026}
}

License

Apache 2.0