ilia-dybal

Qwen-2.5-VL-3b-Instruct-4WA16-GPTQ-Culinary

Deploy Dedicated

README

License: apache-2.0

Model Details

Model Description

Developed by: Ilia Dybal
Funded by: Ilia Dybal
Shared by: Ilia Dybal
Model type: Multimodal Large Visual Language Model (LVLM)
Language(s) (NLP): Russian
License: Apache 2.0
Finetuned from model: Qwen/Qwen2.5-VL-3B-Instruct

Model Sources [optional]

Repository: Qwen/Qwen2.5-VL-3B-Instruct
Paper: https://arxiv.org/pdf/2502.13923

Uses

Direct Use

Structured receipe and ingredients extraction from an image.

Out-of-Scope Use

Medical or clinical dietary diagnostics.
General non-culinary visual analysis (for general VLM tasks, use the base Qwen2.5-VL-3B-Instruct).

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

Option 1: Fast Serving via vLLM (Recommended for Production)

Launch the vLLM server from your terminal:

bash
vllm serve ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary \
    --trust-remote-code \
    --max-model-len 4096 \
    --limit-mm-per-prompt '{"image": 1}'

Query the server using the OpenAI Python Client:

python
import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

def encode_image(image_path):
    with open(image_path, "rb") as f:
        return f"data:image/jpeg;base64,{base64.b64encode(f.read()).decode('utf-8')}"

response = client.chat.completions.create(
    model="ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Analyze this dish and provide the recipe."},
                {"type": "image_url", "image_url": {"url": encode_image("dish.jpg")}}
            ]
        }
    ],
    max_tokens=512
)

print(response.choices[0].message.content)

Option 2: Native Hugging Face Transformers

python
import torch
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

model_id = "ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary"

processor = AutoProcessor.from_pretrained(model_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Prepare multimodal input
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "[https://example.com/dish.jpg](https://example.com/dish.jpg)"},
            {"type": "text", "text": "List the key ingredients in this photo."}
        ]
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=None, return_tensors="pt").to("cuda")

output_ids = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output_ids[0], skip_special_tokens=True))

Training Hyperparameters

Training regime:
- per_device_train_batch_size=16,
- per_device_eval_batch_size=16,
- gradient_accumulation_steps=4,
- num_train_epochs=15,
- gradient_checkpointing=True,
- gradient_checkpointing_kwargs={"use_reentrant": False},
- logging_strategy="steps",
- logging_steps=3,
- eval_strategy="epoch",
- bf16=True,
- logging_first_step=True,
- max_length=2048,
- learning_rate=5e-4,

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: 1xNVIDIA A100(40Gb)
Hours used: 1:21:17 (15 epochs)
Cloud Provider: Google Colab

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Container

Run this model inference with full control and performance in your environment.

Learn more

Model Details

Model Provider

ilia-dybal

Model Tree

Base

Qwen/Qwen2.5-VL-3B-Instruct

Quantized

this model

Input Modalities

TextImage

Output Modalities

Text

Supported Functionality

Dedicated EndpointsContainer

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Details

Model Description

Developed by: Ilia Dybal
Funded by: Ilia Dybal
Shared by: Ilia Dybal
Model type: Multimodal Large Visual Language Model (LVLM)
Language(s) (NLP): Russian
License: Apache 2.0
Finetuned from model: Qwen/Qwen2.5-VL-3B-Instruct

Model Sources [optional]

Repository: Qwen/Qwen2.5-VL-3B-Instruct
Paper: https://arxiv.org/pdf/2502.13923

Uses

Direct Use

Structured receipe and ingredients extraction from an image.

Out-of-Scope Use

Medical or clinical dietary diagnostics.
General non-culinary visual analysis (for general VLM tasks, use the base Qwen2.5-VL-3B-Instruct).

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

Option 1: Fast Serving via vLLM (Recommended for Production)

Launch the vLLM server from your terminal:

bash
vllm serve ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary \
    --trust-remote-code \
    --max-model-len 4096 \
    --limit-mm-per-prompt '{"image": 1}'

Query the server using the OpenAI Python Client:

python
import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

def encode_image(image_path):
    with open(image_path, "rb") as f:
        return f"data:image/jpeg;base64,{base64.b64encode(f.read()).decode('utf-8')}"

response = client.chat.completions.create(
    model="ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Analyze this dish and provide the recipe."},
                {"type": "image_url", "image_url": {"url": encode_image("dish.jpg")}}
            ]
        }
    ],
    max_tokens=512
)

print(response.choices[0].message.content)

Option 2: Native Hugging Face Transformers

python
import torch
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

model_id = "ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary"

processor = AutoProcessor.from_pretrained(model_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Prepare multimodal input
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "[https://example.com/dish.jpg](https://example.com/dish.jpg)"},
            {"type": "text", "text": "List the key ingredients in this photo."}
        ]
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=None, return_tensors="pt").to("cuda")

output_ids = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output_ids[0], skip_special_tokens=True))

Training Hyperparameters

Training regime:
- per_device_train_batch_size=16,
- per_device_eval_batch_size=16,
- gradient_accumulation_steps=4,
- num_train_epochs=15,
- gradient_checkpointing=True,
- gradient_checkpointing_kwargs={"use_reentrant": False},
- logging_strategy="steps",
- logging_steps=3,
- eval_strategy="epoch",
- bf16=True,
- logging_first_step=True,
- max_length=2048,
- learning_rate=5e-4,

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: 1xNVIDIA A100(40Gb)
Hours used: 1:21:17 (15 epochs)
Cloud Provider: Google Colab