Skip to main content
You can use LiteLLM to interact with FriendliAI. This makes migration of existing applications already using LiteLLM particularly easy.

​
How to use

Before you start, ensure you’ve already obtained the FRIENDLI_TOKEN from the Friendli Suite. Add friendliai prefix to your endpoint name for the model parameter.

​
Chat completion

We provide usage examples for each type of endpoint. Choose the one that best suits your needs. You can specify one of the available models for the serverless endpoints. 
import os
from litellm import completion

os.environ['FRIENDLI_TOKEN'] = "YOUR_FRIENDLI_TOKEN"

response = completion(
    model="friendliai/meta-llama-3.3-70b-instruct",
    messages=[
        {"role": "user", "content": "hello from litellm"}
    ],
)
print(response)

​
Chat completion - Streaming

import os
from litellm import completion

os.environ['FRIENDLI_TOKEN'] = "YOUR_FRIENDLI_TOKEN"

response = completion(
    model="friendliai/meta-llama-3.3-70b-instruct",
    messages=[
        {"role": "user", "content": "hello from litellm"}
    ],
    stream=True
)

for chunk in response:
    print(chunk)