You can use OpenAI Node.js SDK to interact with FriendliAI. This makes migration of existing applications already using OpenAI particularly easy.

​
How to use

Before you start, ensure the baseURL and apiKey refer to FriendliAI. Since our products are entirely compatible with OpenAI SDK, now you are good to follow the examples below.

Choose one of the available models for model parameter.

npm i openai

​
Chat Completion

Chat completion API that generates a response from a given conversation.

We provide multiple usage examples. Try to find the best one that aligns with your needs:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.friendli.ai/serverless/v1",
  apiKey: process.env.FRIENDLI_TOKEN,
});

async function main() {
  const completion = await client.chat.completions.create({
    model: "meta-llama-3.1-8b-instruct",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "Hello!" },
    ],
  });

  console.log(completion.choices[0]);
}

main();

​
Tool assisted chat completion

This feature is in Beta and available only on the Serverless Endpoints.

Using tool assisted chat completion API, models can utilize built-in tools prepared for tool calls, enhancing its capability to provide more comprehensive and actionable responses.

Available tools are listed here.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.friendli.ai/serverless/v1",
  apiKey: process.env.FRIENDLI_TOKEN,
});

async function main() {
  const messages = [
    {
      role: "user",
      content:
        "What is the current average home price in New York City, and if I put 15% down, how much will my mortgage be?",
    },
  ];

  const tools = [{ type: "code:python-interpreter" }, { type: "web:search" }];

  const completion = await client.chat.completions.create({
    model: "meta-llama-3.1-8b-instruct",
    messages: messages,
    tools: tools,
    tool_choice: "auto",
    stream: true,
  });

  for await (const chunk of completion) {
    if (chunk.choices === undefined) {
      console.log(`event: ${chunk.event}, data: ${JSON.stringify(chunk.data)}`);
    } else {
      console.log(chunk.choices[0].delta.content);
    }
  }
}

main();

Was this page helpful?