The Vision feature is available when the model supports vision capabilities.

Friendli is equipped with a new Vision feature that can understand and analyze images, opening up exciting possibilities for multimodal interactions. This guide explains how to work with images in Friendli, including best practices and code examples.

​
How to Use Vision

Utilize Friendli’s Vision features through the following:

  • Select and test a vision model at friendli.ai/playground.
  • Use the API to process images and receive the model’s responses, referring to the methods described in this document.

​
Using the API

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.friendli.ai/dedicated/v1",
    api_key=os.environ.get("FRIENDLI_TOKEN"),
)

image_url = "https://upload.wikimedia.org/wikipedia/commons/9/9e/Ours_brun_parcanimalierpyrenees_1.jpg"

completion = client.chat.completions.create(
    # Replace YOUR_ENDPOINT_ID with the ID of your endpoint, e.g. "zbimjgovmlcb"
    model="YOUR_ENDPOINT_ID",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What kind of animal is shown in the image?",
                },
                {"type": "image_url", "image_url": {"url": image_url}},
            ],
        },
    ],
)

print(completion.choices[0].message.content)

Was this page helpful?