What is Tool Assisted API?
Tool Assisted API enhances a model’s capabilities by integrating tools that extend its functionality beyond simple conversational interactions. By using this API, the model becomes more dynamic, providing more comprehensive and actionable responses. Currently, Friendli Serverless Endpoints supports a variety of built-in tools specifically designed for Chat Completion tasks.What is Chat Completion?
Chat completion refers to a model’s ability to generate responses in a conversation. Given a sequence of messages or conversation turns, the model processes the input and generates a response based on its internal knowledge and training data.- Example:
- User: “What is the capital of France?”
- Model: “The capital of France is Paris.”
Is Chat Completion Different from Tool Assisted Chat Completion?
Yes, Tool Assisted Chat Completion goes beyond basic chat completion by integrating external tools to enhance the conversation. This allows the model to access real-time data, perform specific tasks, and interact with external systems in ways that chat completion alone cannot achieve.- Example:
- User: “What is the weather today?”
- Model without Tool Access: Relies on pre-learned information, potentially giving outdated or generalized answers.
- Model with Tool Access: Calls a weather API to retrieve live data and responds: “The weather today in New York is 72°F with clear skies.”
- Example:
- User: “Can you extract the text from this document?” (provides a file)
- Model without Tool Access: “I cannot extract data from files directly.”
- Model with Tool Access: Extracts the text from the provided file and responds: “Using the
file:texttool, I’ve extracted the following text: [Text from the file].”
Benefits of Tool Assisted Chat Completion
Tool Assisted Chat Completion offers several advantages over basic chat completion:- Real-Time Data Access: The model can pull live information.
- Extended Capabilities: The model can perform complex tasks like running calculations, executing code, extracting text from files, and interacting with databases and APIs.
Comparison: Chat Completion vs. Tool Assisted Chat Completion
| Feature | Chat Completion | Tool Assisted Chat Completion |
|---|---|---|
| Response Type | Based on internal knowledge | Uses external tools for enhanced, real-time responses |
| Capabilities | Limited to pre-learned knowledge | Can interact with tools for data retrieval and task execution |
| Example | ”What is the weather today?” (general knowledge) | “What is the weather today?” (live API result) |
| Use Cases | General conversation and Q&A | Complex tasks like real-time updates, data analysis, file processing |
Integrated Tools
Tool Assisted API can also make use of integrated tools, which operate in the same way as built-in tools but require a connection to external services. Because they leverage specialized APIs or platforms, integrated tools are more likely to be production-ready compared to built-in tools, offering higher performance and broader functionality.linkup:search 
Description:
Linkup seamlessly integrates with models served by FriendliAI to provide real-time web search capabilities. This enables AI applications to retrieve up-to-date facts, events, and information beyond the model’s training data, complete with accurate citations.
Grounding responses in real-time data, Linkup improves precision, accuracy and factuality while delivering production-ready state-of-the-art web information retrieval.
When Used:
Automatically called when you need to retrieve current information from the web, such as recent news, real-time data, or up-to-date facts.
This tool is particularly useful for tasks requiring accurate and reliable web search results.
This tool requires integration setup for SDK/API usage. You can obtain your Linkup API key from app.linkup.so and integrate it in the Friendli Suite > Personal Settings > Integrations.For more details, see the Linkup integration guide.
Built-In Tools
Tool Assisted API automatically selects the best tool to perform an action based on user input when a specific tool is enabled. Built-in tools are available without any external integration, making them free to use and instantly accessible. See the list below for available built-in tools.web:search
Description:
Retrieves information from the web based on search queries. It fetches information based on keywords and helps gather
knowledge or insights from online sources.
When Used:
Automatically called when you ask questions or seek information that requires external research or the latest data from the web.However, compared to specialized web search tools that offer advanced reasoning and enhanced accuracy with citations, this built-in tool may have limitations for complex queries and production applications.
math:calculator
Description:
Performs basic arithmetic operations like addition, subtraction, multiplication, division, and more complex calculations like and square roots or exponents.
It is useful for any tasks requiring mathematical computation.
When Used:
Automatically called when mathematical expressions or calculations are required.Whether you’re solving equations, calculating percentages, or handling financial calculations, this tool performs the task for you.
code:python-interpreter
Description:
Executes Python code directly within the platform for custom scripts, data processing, or automation.
You can run Python scripts, test snippets of code, or automate tasks through coding logic.
When Used:
Automatically called when tasks involve writing or running Python scripts, such as custom data manipulations or logic-based automation.
Conclusion
- Chat Completion: Best for general conversations that rely on the model’s pre-existing knowledge.
- Tool Assisted Chat Completion: Ideal for real-time, dynamic tasks and more advanced interactions, leveraging external tools to enhance functionality.