import torch
from transformers import AutoModelForVision2Seq, AutoProcessor
from peft import PeftModel
base_model_id = "unsloth/Qwen2.5-VL-7B-Instruct"
base_model = AutoModelForVision2Seq.from_pretrained(
base_model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "kiel2/KielLens-VQA")
processor = AutoProcessor.from_pretrained("kiel2/KielLens-VQA")
Training Procedure
The model was fine-tuned using the Unsloth framework and TRL library with the following configurations:
Quantization: 4-bit (via NF4) for memory efficiency and high-speed inference.
Target Modules: Vision and attention layers optimised for multimodal token mapping.
Optimization: Paged AdamW 32-bit.
Precision: Mixed precision (bf16).
Limitations and Ethical Considerations
Input Sensitivity: Performance may vary depending on image resolution, lighting quality, and framing.
Generalization: While optimized for robust visual reasoning, the model should be deployed with human oversight for mission-critical visual analytics tasks.