Skip to main content
Friendli provides OpenAI-compatible tool calling with two core guarantees:
  • Broad model coverage: Works across most chat‑capable models. No custom parsers required.
  • High accuracy: Ensures reliable tool-call response that aligns with your provided schemas.

​
What is Tool Calling?

Tool calling (also called function calling) connects LLMs to external systems, enabling real‑time data access and action execution—capability essential for agentic workflows.
Function calling

​
Broad Model Coverage

Friendli supports tool calling for a wide range of open‑source and commercial models.
You can browse available models on our Models page and try them out with the Playground.

​
Tool Calling with Friendli

Friendli supports tool calling for a wide range of open‑source and commercial models.

​
Tool Calling Parameters

To enable tool calling, use the tools, tool_choice, and parallel_tool_calls parameters.
ParameterDescriptiondefault
toolsThe list of tool objects that define the functions the model can call.-
tool_choiceDetermines the tool calling behavior of the model.auto
parallel_tool_callsWhether to let the model issue tool calls in parallel.True
By default, the model decides whether to call a function and which one to use. With the tool_choice parameter, you can explicitly instruct the model to use a specific function.
  • none: Disable the use of tools.
  • auto: Enable the model to decide whether to use tools and which ones to use.
  • required: Force the model to use a tool, but the model chooses which one.
  • Named tool choice: Force the model to use a specific tool. It must be in the following format: 
    {
  "type": "function",
  "function": {
    "name": "get_current_weather" // The function name you want to specify
  }
}

​
Response Schema

Friendli follows the OpenAI function calling schema. Tool calls are returned in choices[].message.tool_calls[] with each item containing a function.name and JSON‑stringified function.arguments. After executing a tool, append a new message with role: tool, the matching tool_call_id, and the tool result in content.

​
Simple Example

The example below walks through five steps:
  1. Define a tool (get_weather) that retrieves weather information.
  2. Ask a question that triggers tool use.
  3. Let the model select the tool.
  4. Execute the tool.
  5. Generate the final answer using the tool result.
Open In Colab
1

Tool Definition

Define a function that the model can call (get_weather) with a JSON Schema.
The function requires the following parameters:
  • location: The location to look up weather information for.
  • date: The date to look up weather information for.
This definition is included in the tools array and passed to the model.
tools = [
  {
      "type": "function",
      "function": {
          "name": "get_weather",
          "parameters": {
              "type": "object",
              "properties": {
                  "location": {"type": "string"},
                  "date": {"type": "string", "format": "date"}
              },
          },
      },
  }
]
2

Calling the model

When a user asks a question, this request is passed to the model as a messages array.
For example, the request “What’s the weather like in Paris today?” would be passed as:
from datetime import datetime

today = datetime.now()
messages = [
    {"role": "system", "content": f"You are a helpful assistant. today is {today}."},
    {"role": "user", "content": "What's the weather like in Paris today?"}
]
Call the model using the tools and messages defined above.
import os
from openai import OpenAI

token = os.getenv("FRIENDLI_TOKEN") or "<YOUR_FRIENDLI_TOKEN>"

client = OpenAI(
    base_url = "https://api.friendli.ai/serverless/v1",
    api_key = token
)

completion = client.chat.completions.create(
  model="meta-llama-3.1-8b-instruct",
  messages=messages,
  tools=tools,
)

print(completion.choices[0].message.tool_calls)
3

Execution tool

The API caller runs the tool based on the function call information of the model.
For example, the get_weather function is executed as follows:
import json
import random

def get_weather(location: str, date: str):
    temperature = random.randint(60, 80)
    return {"temperature": temperature, "forecast": "sunny"}

tool_call = completion.choices[0].message.tool_calls[0]

tool_response = locals()[tool_call.function.name](**json.loads(tool_call.function.arguments))
print(tool_response)
Result:
{'temperature': 65, 'forecast': 'sunny'}
4

Adding tool responses

Add the tool’s response to the messages array and pass it back to the model.
  1. Append tool call information
  2. Append the tool’s execution result
This ensures the model has all the necessary information to generate a response.
model_response = completion.choices[0].message

# Append the response from the model
messages.append(
    {
        "role": model_response.role,
        "tool_calls": [
            tool_call.model_dump()
            for tool_call in model_response.tool_calls
        ]
    }
)

# Append the response from the tool
messages.append(
    {
        "role": "tool",
        "content": json.dumps(tool_response),
        "tool_call_id": tool_call.id
    }
)

print(json.dumps(messages, indent=2))
5

Generating the final response

The model generates the final response based on the tool’s output:
next_completion = client.chat.completions.create(
    model="meta-llama-3.1-8b-instruct",
    messages=messages,
    tools=tools
)

print(next_completion.choices[0].message.content)
Final output:
According to the forecast, it's going to be a sunny day in Paris with a temperature of 65 degrees.

​
Advanced Examples

Follow the following blog posts to learn more about how to use tool calling with Friendli:
⌘I