GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
ilia-dybal
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
ilia-dybal
Model Tree
Qwen/Qwen2.5-VL-3B-Instruct
Input Modalities
Output Modalities
Supported Functionality
Qwen2.5-VL-3B-Instruct).[More Information Needed]
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
Use the code below to get started with the model.
Launch the vLLM server from your terminal:
bash
vllm serve ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary \ --trust-remote-code \ --max-model-len 4096 \ --limit-mm-per-prompt '{"image": 1}'Query the server using the OpenAI Python Client:
python
import base64from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") def encode_image(image_path): with open(image_path, "rb") as f: return f"data:image/jpeg;base64,{base64.b64encode(f.read()).decode('utf-8')}" response = client.chat.completions.create( model="ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary", messages=[ { "role": "user", "content": [ {"type": "text", "text": "Analyze this dish and provide the recipe."}, {"type": "image_url", "image_url": {"url": encode_image("dish.jpg")}} ] } ], max_tokens=512) print(response.choices[0].message.content)python
import torchfrom transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration model_id = "ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary" processor = AutoProcessor.from_pretrained(model_id)model = Qwen2_5_VLForConditionalGeneration.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto") # Prepare multimodal inputmessages = [ { "role": "user", "content": [ {"type": "image", "image": "[https://example.com/dish.jpg](https://example.com/dish.jpg)"}, {"type": "text", "text": "List the key ingredients in this photo."} ] }] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[text], images=None, return_tensors="pt").to("cuda") output_ids = model.generate(**inputs, max_new_tokens=512)print(processor.decode(output_ids[0], skip_special_tokens=True))Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
vllm serve ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary \ --trust-remote-code \ --max-model-len 4096 \ --limit-mm-per-prompt '{"image": 1}'import base64from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") def encode_image(image_path): with open(image_path, "rb") as f: return f"data:image/jpeg;base64,{base64.b64encode(f.read()).decode('utf-8')}" response = client.chat.completions.create( model="ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary", messages=[ { "role": "user", "content": [ {"type": "text", "text": "Analyze this dish and provide the recipe."}, {"type": "image_url", "image_url": {"url": encode_image("dish.jpg")}} ] } ], max_tokens=512) print(response.choices[0].message.content)import torchfrom transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration model_id = "ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary" processor = AutoProcessor.from_pretrained(model_id)model = Qwen2_5_VLForConditionalGeneration.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="auto") # Prepare multimodal inputmessages = [ { "role": "user", "content": [ {"type": "image", "image": "[https://example.com/dish.jpg](https://example.com/dish.jpg)"}, {"type": "text", "text": "List the key ingredients in this photo."} ] }] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[text], images=None, return_tensors="pt").to("cuda") output_ids = model.generate(**inputs, max_new_tokens=512)print(processor.decode(output_ids[0], skip_special_tokens=True))