@font-face {
  font-family: "Saans-Light";
  src:
    url("https://docs-assets.friendli.ai/fonts/Saans-Light.woff2")
      format("woff2"),
    url("https://docs-assets.friendli.ai/fonts/Saans-Light.woff") format("woff");
  font-weight: 300;
  font-style: normal;
  font-display: swap;
}
@font-face {
  font-family: "Saans-Regular";
  src:
    url("https://docs-assets.friendli.ai/fonts/Saans-Regular.woff2")
      format("woff2"),
    url("https://docs-assets.friendli.ai/fonts/Saans-Regular.woff")
      format("woff");
  font-weight: 400;
  font-style: normal;
  font-display: swap;
}
@font-face {
  font-family: "Saans-Medium";
  src:
    url("https://docs-assets.friendli.ai/fonts/Saans-Medium.woff2")
      format("woff2"),
    url("https://docs-assets.friendli.ai/fonts/Saans-Medium.woff")
      format("woff");
  font-weight: 500;
  font-style: normal;
  font-display: swap;
}
@font-face {
  font-family: "Saans-SemiBold";
  src:
    url("https://docs-assets.friendli.ai/fonts/Saans-SemiBold.woff2")
      format("woff2"),
    url("https://docs-assets.friendli.ai/fonts/Saans-SemiBold.woff")
      format("woff");
  font-weight: 600;
  font-style: normal;
  font-display: swap;
}
@font-face {
  font-family: "Saans-Bold";
  src:
    url("https://docs-assets.friendli.ai/fonts/Saans-Bold.woff2")
      format("woff2"),
    url("https://docs-assets.friendli.ai/fonts/Saans-Bold.woff") format("woff");
  font-weight: 700;
  font-style: normal;
  font-display: swap;
}
@font-face {
  font-family: "Saans-Heavy";
  src:
    url("https://docs-assets.friendli.ai/fonts/Saans-Heavy.woff2")
      format("woff2"),
    url("https://docs-assets.friendli.ai/fonts/Saans-Heavy.woff") format("woff");
  font-weight: 800;
  font-style: normal;
  font-display: swap;
}

body {
  font-family: "Saans-Regular", sans-serif;
  letter-spacing: 0.01em;
}

.tracking-tighter {
  letter-spacing: -0.04em;
}
.tracking-tight {
  letter-spacing: 0.01em;
}
.tracking-normal {
  letter-spacing: 0.01em;
}
.tracking-wide {
  letter-spacing: 0.01em;
}
.tracking-wider {
  letter-spacing: 0.05em;
}
.tracking-widest {
  letter-spacing: 0.1em;
}
.prose
  :where(h1, h2, h3):not(:where([class~="not-prose"], [class~="not-prose"] *)) {
  letter-spacing: 0;
}

.card > div > div:last-child > h2 {
  font-size: 1.25rem;
}
.card > div > div:last-child > div {
  margin-top: 1rem;
}

.font-medium:not(.font-mono):not(.font-mono *) {
  font-family: "Saans-Medium", sans-serif;
}

h1:not(.font-mono):not(.font-mono *),
h2:not(.font-mono):not(.font-mono *),
h3:not(.font-mono):not(.font-mono *),
h4:not(.font-mono):not(.font-mono *),
h5:not(.font-mono):not(.font-mono *),
strong:not(.font-mono):not(.font-mono *),
.font-semibold:not(.font-mono):not(.font-mono *),
.prose
  :where(a):not(:where([class~="not-prose"], [class~="not-prose"] *)):not(
    .font-mono
  ):not(.font-mono *) {
  font-family: "Saans-SemiBold", sans-serif;
}

#header > div:not(:first-child) {
  display: none;
}

#topbar-cta-button > .font-medium {
  font-family: "Saans-Medium", sans-serif;
}

.border-gray-100 {
  border-color: #d9e2ec;
}

:root {
  --gray-300: 159 179 200;
}

.dark\:bg-codeblock div[role="tablist"] button[role="tab"] {
  /* prettier-ignore */
  font-family:
    var(--font-jetbrains-mono), ui-monospace, SFMono-Regular, Menlo, Monaco,
    Consolas, "Liberation Mono", "Courier New", monospace;
}

.rounded-border-box {
  padding: 28px 44px;
  border: 1px solid #d9e3ee;
  border-radius: 8px;
  overflow: hidden;

  img {
    margin: 0 auto;
    border: 1px solid #e7eef4;
    border-radius: 8px;
  }

  p {
    margin: 12px auto -12px;
    text-align: center;
    word-break: break-all;
  }
}

.dark {
  .rounded-border-box {
    border: 1px solid #343842;

    img {
      border: 1px solid #212832;
    }
  }
}


import * as mdx from "eslint-plugin-mdx";

export default [
  {
    files: ["**/*.mdx"],
    ...mdx.flat,
    processor: mdx.createRemarkProcessor({
      lintCodeBlocks: true,
      languageMapper: {},
    }),
    rules: {
      "no-unused-expressions": "off",
    },
  },
  {
    files: ["**/*.mdx"],
    ...mdx.flatCodeBlocks,
    rules: {
      ...mdx.flatCodeBlocks.rules,
      "no-var": "error",
      "prefer-const": "error",
    },
  },
];


QuickStarts

Explore APIs

Get started with FriendliAI products and explore APIs.

Overview

Friendli Documentation

Friendli Docs

Friendli Suite empowers you to explore generative AI with three solutions: Serverless Endpoints for quick access to open-source models, Dedicated Endpoints for deploying custom models on dedicated GPUs, and Containers for secure, on-premise control. Powered by the optimized Friendli Engine, each option ensures fast, cost-efficient AI serving for text, code, and image generation.

Introduction

Unleash the Power of Generative AI with Friendli Suite: Your End-to-End Solution

Learn how to manage credentials in Friendli Suite, including using personal access tokens for authentication and authorization.

Personal Access Tokens

Dive into the characteristics of popular Text Generation Models (TGMs) available on Friendli.

Text Generation

Text Generation Models

Dive into the characteristics of popular Image Generation Models available on Friendli Dedicated Endpoints.

Image Generation

Visualizing Ideas with Friendli: A Guide to Image Generation

Guide to using Friendli's Vision feature for image analysis. Covers usage via Playground and API (URL & Base64 examples).

Vision

Vision: Image understanding with Friendli

Guide to using Friendli's Audio and Speech feature for audio analysis and transcription. Covers usage via Playground and API (URL & Base64 examples).

Audio and Speech

Audio and Speech: Converting Audio to Text

Learn how to do OpenAI compatible function calling on Friendli Serverless Endpoints.

Function Calling

Generate structured outputs using FriendliAI's Structured Outputs feature.

Structured Outputs

Friendli Serverless Endpoints is compatible with the OpenAI API standard through the Python API Libraries and the Node API Libraries. Friendli Dedicated Endpoints and Friendli Container are also OpenAI API compatible.

OpenAI Compatibility

Fine-tune Vision Language Models (VLMs) on Friendli Dedicated Endpoints using datasets.

How to Fine-tune Vision Language Models (VLMs)

Build AI agents with Friendli Serverless Endpoints using tool calling for dynamic, real-time interactions with LLMs.

Tool calling with Serverless Endpoints

Hands-on tutorial for launching and deploying LLMs using Friendli Dedicated Endpoints with Weights & Biases artifacts through webhook automation.

Deploy from W&B Registry with Webhook

Build an AI agent with LangChain and Friendli Serverless Endpoints, integrating tool calling for dynamic and efficient responses.

Build an agent with LangChain

Build and deploy smart AI agents with Friendli Serverless Endpoints and Gradio in under 50 lines.

Build an agent with Gradio

Go Playground with Next.js

Chat docs with LangChain

Chat docs with MongoDB

RAG app with LlamaIndex

Interact with Friendli AI services using the official Python SDK for seamless integration with your applications.

Python SDK

Friendli Python SDK

Effortlessly integrate FriendliAI models into your projects with support for popular SDKs and frameworks.

Friendli Integrations

Easily integrate FriendliAI models with the Vercel AI SDK, supporting serverless, dedicated, and fine-tuned endpoints.

Vercel AI SDK

Easily integrate large language models with the LlamaIndex SDK, featuring FriendliAI for seamless interaction.

LlamaIndex

LiteLLM SDK supports all FriendliAI models, offering easy integration with serverless, dedicated, and fine-tuned endpoints.

LiteLLM

OpenAPI reference of Friendli Suite API. You can interact with the API through HTTP requests from any language.

Friendli Suite API Reference

OpenAPI reference of Friendli Dedicated Endpoints API.

Dedicated overview

Given a list of messages forming a conversation, the model generates a response.

Chat completions

Dedicated chat completions

Generate text based on the given text prompt.

Completions

Dedicated completions

By giving a text input, generate a tokenized output of token IDs.

Tokenization

Dedicated tokenization

By giving a list of tokens, generate a detokenized output text string.

Detokenization

Dedicated detokenization

Given a description, the model generates image(s).

Image generations

Dedicated image generations (Beta)

Represents a streamed chunk of a chat completions response returned by model, based on the provided input.

Chat completions chunk object

Dedicated chat completions chunk object

Represents a streamed chunk of a completions response returned by model, based on the provided input.

Completions chunk object

Dedicated completions chunk object

List endpoints

Dedicated list endpoints (Beta)

Given an endpoint ID, return its specification.

Retrieve an endpoint specification

Dedicated get endpoint (Beta)

Given an endpoint ID, return its current status.

Retrieve an endpoint status

Dedicated get endpoint status (Beta)

Create a Dedicated Endpoint deployment for a Hugging Face model.

Create an endpoint with Hugging Face model

Dedicated create endpoint (Beta)

Create an endpoint from Weights & Biases artifact.

Create endpoint from W&B artifact

Dedicated create endpoint from W&B artifact

Update a Dedicated Endpoint deployment with new configuration.

Update an endpoint with new configuration

Dedicated update endpoint (Beta)

Terminate an endpoint

Dedicated terminate endpoint (Beta)

Restart a failed or terminated Dedicated Endpoint.

Restart an endpoint

Dedicated restart endpoint (Beta)

OpenAPI reference of Friendli Serverless Endpoints API.

Serverless overview

Serverless chat completions

Given a list of messages forming a conversation, the model generates a response. Additionally, the model can utilize built-in tools for tool calls, enhancing its capability to provide more comprehensive and actionable responses.

Tool assisted chat completions

Serverless tool assisted chat completions (Beta)

Serverless completions

Serverless tokenization

Serverless detokenization

Serverless chat completions chunk object

Represents a streamed chunk of a tool assisted chat completions response returned by model, based on the provided input.

Tool assisted chat completions chunk object

Serverless tool assisted chat completions chunk object (Beta)

Serverless completions chunk object

OpenAPI reference of Friendli Container API.

Container overview

Container chat completions

Container completions

Container tokenization

Container detokenization

Given a description, the model generates image.

Container image generations (Beta)

Container chat completions chunk object

Container completions chunk object

OpenAPI reference of Friendli Dataset API.

Dataset overview

List datasets

List datasets (Beta)

List versions

List versions (Beta)

List splits

List splits (Beta)

List samples

List samples (Beta)

Get dataset info

Get dataset info (Beta)

Get version info

Get version info (Beta)

Get split info

Get split info (Beta)

Create a new dataset

Create a new dataset (Beta)

Create a new version

Create a new version (Beta)

Create a new split

Create a new split (Beta)

Add samples

Add samples (Beta)

Delete samples

Delete samples (Beta)

Update samples

Update samples (Beta)

Delete a version

Delete a version (Beta)

Delete a dataset

Delete a dataset (Beta)

Delete a split

Delete a split (Beta)

File overview

Get file info

Get file info (Beta)

Get file download URL

Get file download URL (Beta)

Initiate file upload

Initiate file upload (Beta)

Complete file upload

Complete file upload (Beta)

Install friendli-client package to access advanced features for AI integration. Supports Python 3.8+, with options for machine learning libraries and Hugging Face checkpoint conversion.

Installation

Sign in to Friendli using the command line interface.

friendli login

Sign out to Friendli using the command line interface.

friendli logout

Check the installed package version of Friendli using the command line interface.

friendli version

Show my user information of Friendli using the command line interface.

friendli whoami

Create and deploy new endpoints with the Friendli API. Customize with model selection, GPU configuration, and more to efficiently serve your machine learning models.

friendli endpoint create

Get detailed information about a specific endpoint using the Friendli API.

friendli endpoint get

View all your deployed endpoints with the Friendli API. Easily list endpoints for efficient model management.

friendli endpoint list

Terminate a running endpoint with the Friendli API using the endpoint ID. Easily manage and stop your deployed models when needed.

friendli endpoint terminate

Convert Hugging Face model checkpoints to Friendli format for deployment. Includes options for quantization, data type selection, and model optimization using the Friendli API.

friendli model convert

View all available models with the Friendli API. Easily list models to streamline your deployment and optimization processes.

friendli model list

List all accessible projects with the Friendli API. Easily manage your available projects for efficient workflow management.

friendli project list

Switch between project contexts using the Friendli API. Quickly change the active project by providing the project ID for smooth workflow management.

friendli project switch

View all available teams with the Friendli API. Easily list teams for project organization.

friendli team list

Switch between team contexts using the Friendli API. Quickly change the active team by providing the team ID for efficient collaboration and management.

Get Started

Core Concepts

Products

Tutorials

Friendli Documentation

QuickStarts

Friendli Dedicated Endpoints QuickStart

Friendli Serverless Endpoints QuickStart

Friendli Container QuickStart

SDK

Python SDK

Friendli Integrations

Explore APIs

Chat completions

Tool assisted chat completions (Beta)

Get Started

Core Concepts

Products

Tutorials

​QuickStarts

Friendli Dedicated Endpoints QuickStart

Friendli Serverless Endpoints QuickStart

Friendli Container QuickStart

​SDK

Python SDK

Friendli Integrations

​Explore APIs

Chat completions

Tool assisted chat completions (Beta)

QuickStarts

SDK

Explore APIs