Tool Assisted API

​

What is Tool Assisted API?

Tool Assisted API enhances a model’s capabilities by integrating tools that extend its functionality beyond simple conversational interactions. By using this API, the model becomes more dynamic, providing more comprehensive and actionable responses. Currently, Friendli Serverless Endpoints supports a variety of built-in tools specifically designed for Chat Completion tasks.

---

​

What is Chat Completion?

Chat completion refers to a model’s ability to generate responses in a conversation. Given a sequence of messages or conversation turns, the model processes the input and generates a response based on its internal knowledge and training data.

• Example:
  • User: “What is the capital of France?”
  • Model: “The capital of France is Paris.”

However, chat completion has its limitations—it is restricted to the knowledge the model has learned during its training and cannot access real-time or external data.

---

​

Is Chat Completion Different from Tool Assisted Chat Completion?

Yes, Tool Assisted Chat Completion goes beyond basic chat completion by integrating external tools to enhance the conversation. This allows the model to access real-time data, perform specific tasks, and interact with external systems in ways that chat completion alone cannot achieve.

• Example:
  • User: “What is the weather today?”
  • Model without Tool Access: Relies on pre-learned information, potentially giving outdated or generalized answers.
  • Model with Tool Access: Calls a weather API to retrieve live data and responds: “The weather today in New York is 72°F with clear skies.”

With tool access, the model provides a more accurate and up-to-date response. Additionally, some tasks—such as file processing or complex calculations—cannot be performed by the model alone but can be handled with the help of tools.

• Example:
  • User: “Can you extract the text from this document?” (provides a file)
  • Model without Tool Access: “I cannot extract data from files directly.”
  • Model with Tool Access: Extracts the text from the provided file and responds: “Using the file:text tool, I’ve extracted the following text: [Text from the file].”

When no tools are specified, the model will respond using only its internal knowledge.

---

​

Benefits of Tool Assisted Chat Completion

Tool Assisted Chat Completion offers several advantages over basic chat completion:

• Real-Time Data Access: The model can pull live information.
• Extended Capabilities: The model can perform complex tasks like running calculations, executing code, extracting text from files, and interacting with databases and APIs.

---

​

Comparison: Chat Completion vs. Tool Assisted Chat Completion

Feature	Chat Completion	Tool Assisted Chat Completion
Response Type	Based on internal knowledge	Uses external tools for enhanced, real-time responses
Capabilities	Limited to pre-learned knowledge	Can interact with tools for data retrieval and task execution
Example	”What is the weather today?” (general knowledge)	“What is the weather today?” (live API result)
Use Cases	General conversation and Q&A	Complex tasks like real-time updates, data analysis, file processing

---

​

Integrated Tools

Tool Assisted API can also make use of integrated tools, which operate in the same way as built-in tools but require a connection to external services. Because they leverage specialized APIs or platforms, integrated tools are more likely to be production-ready compared to built-in tools, offering higher performance and broader functionality.

​

linkup:search

Description: Linkup seamlessly integrates with models served by FriendliAI to provide real-time web search capabilities. This enables AI applications to retrieve up-to-date facts, events, and information beyond the model’s training data, complete with accurate citations. Grounding responses in real-time data, Linkup improves precision, accuracy and factuality while delivering production-ready state-of-the-art web information retrieval. When Used: Automatically called when you need to retrieve current information from the web, such as recent news, real-time data, or up-to-date facts. This tool is particularly useful for tasks requiring accurate and reliable web search results.

---

​

Built-In Tools

Tool Assisted API automatically selects the best tool to perform an action based on user input when a specific tool is enabled. Built-in tools are available without any external integration, making them free to use and instantly accessible. See the list below for available built-in tools.

​

web:search

Description: Retrieves information from the web based on search queries. It fetches information based on keywords and helps gather knowledge or insights from online sources. When Used: Automatically called when you ask questions or seek information that requires external research or the latest data from the web.
However, compared to specialized web search tools that offer advanced reasoning and enhanced accuracy with citations, this built-in tool may have limitations for complex queries and production applications.

​

math:calculator

Description: Performs basic arithmetic operations like addition, subtraction, multiplication, division, and more complex calculations like and square roots or exponents. It is useful for any tasks requiring mathematical computation. When Used: Automatically called when mathematical expressions or calculations are required.
Whether you’re solving equations, calculating percentages, or handling financial calculations, this tool performs the task for you.

​

code:python-interpreter

Description: Executes Python code directly within the platform for custom scripts, data processing, or automation. You can run Python scripts, test snippets of code, or automate tasks through coding logic. When Used: Automatically called when tasks involve writing or running Python scripts, such as custom data manipulations or logic-based automation.

​

Conclusion

• Chat Completion: Best for general conversations that rely on the model’s pre-existing knowledge.
• Tool Assisted Chat Completion: Ideal for real-time, dynamic tasks and more advanced interactions, leveraging external tools to enhance functionality.

---

​

Explore APIs

To get started with Tool Assisted Chat Completion, follow this tutorial: Tool calling with Serverless Endpoints. For more details, check out the API Reference documentations below: