Tool Assisted API (Beta)
Tool Assisted API enhances a model’s capabilities by integrating tools that extend its functionality beyond simple conversational interactions. By using this API, the model becomes more dynamic, providing more comprehensive and actionable responses. Currently, Friendli Serverless Endpoints supports a variety of built-in tools specifically designed for Chat Completion tasks.
What is Tool Assisted API?
Tool Assisted API enhances a model’s capabilities by integrating tools that extend its functionality beyond simple conversational interactions. By using this API, the model becomes more dynamic, providing more comprehensive and actionable responses.
Currently, Friendli Serverless Endpoints supports a variety of built-in tools specifically designed for Chat Completion tasks.
What is Chat Completion?
Chat completion refers to a model’s ability to generate responses in a conversation. Given a sequence of messages or conversation turns, the model processes the input and generates a response based on its internal knowledge and training data.
- Example:
- User: “What is the capital of France?”
- Model: “The capital of France is Paris.”
However, chat completion has its limitations—it is restricted to the knowledge the model has learned during its training and cannot access real-time or external data.
Is Chat Completion Different from Tool Assisted Chat Completion?
Yes, Tool Assisted Chat Completion goes beyond basic chat completion by integrating external tools to enhance the conversation. This allows the model to access real-time data, perform specific tasks, and interact with external systems in ways that chat completion alone cannot achieve.
- Example:
- User: “What is the weather today?”
- Model without Tool Access: Relies on pre-learned information, potentially giving outdated or generalized answers.
- Model with Tool Access: Calls a weather API to retrieve live data and responds: “The weather today in New York is 72°F with clear skies.”
With tool access, the model provides a more accurate and up-to-date response.
Additionally, some tasks—such as file processing or complex calculations—cannot be performed by the model alone but can be handled with the help of tools.
- Example:
- User: “Can you extract the text from this document?” (provides a file)
- Model without Tool Access: “I cannot extract data from files directly.”
- Model with Tool Access: Extracts the text from the provided file and responds: “Using the
file:texttool, I’ve extracted the following text: [Text from the file].”
When no tools are specified, the model will respond using only its internal knowledge.
Benefits of Tool Assisted Chat Completion
Tool Assisted Chat Completion offers several advantages over basic chat completion:
- Real-Time Data Access: The model can pull live information.
- Extended Capabilities: The model can perform complex tasks like running calculations, executing code, extracting text from files, and interacting with databases and APIs.
Comparison: Chat Completion vs. Tool Assisted Chat Completion
| Feature | Chat Completion | Tool Assisted Chat Completion |
|---|---|---|
| Response Type | Based on internal knowledge | Uses external tools for enhanced, real-time responses |
| Capabilities | Limited to pre-learned knowledge | Can interact with tools for data retrieval and task execution |
| Example | ”What is the weather today?” (general knowledge) | “What is the weather today?” (live API result) |
| Use Cases | General conversation and Q&A | Complex tasks like real-time updates, data analysis, file processing |
Built-In Tools
Tool Assisted API automatically selects the best tool to perform an action based on user input when a specific tool is enabled.
These tools can handle various operations, such as calculations, statistical analysis, web search, file content extraction, and code execution.
Below is a more detailed description of the available tools in Tool Assisted API and when they are typically used:
math:calculator
Description:
Performs basic arithmetic operations like addition, subtraction, multiplication, division, and more complex calculations like and square roots or exponents. It is useful for any tasks requiring mathematical computation.
When Used:
Automatically called when mathematical expressions or calculations are required. Whether you’re solving equations, calculating percentages, or handling financial calculations, this tool performs the task for you.
math:statistics
Description:
Performs statistical analysis, including calculating mean, median, mode, standard deviation, and correlations. It is tailored for situations where you need to analyze or interpret numeric datasets to understand trends or patterns.
When Used:
Automatically called when analyzing numeric data or generating insights from datasets, like summarizing survey results, or calculating probabilities.
math:calendar
Description:
Handles date-related data, such as calculating date differences or finding specific days in the past or future. It is effective in managing and manipulating calendar-based information.
When Used:
Automatically called when operations involving dates or time spans are required, like finding how many days remain until an event. For example, figuring out how many days are left until an event, determining the day of the week for a specific date, or calculating the duration between two dates.
web:search
Description:
Retrieves information from the web based on search queries. It fetches information based on keywords and helps gather knowledge or insights from online sources.
When Used:
Automatically called when you ask questions or seek information that requires external research or the latest data from the web. Whether it is looking up definitions, recent news, or general web searches, this tool handles such tasks effectively.
web:url
Description:
Extracts specific data from a given website. You can provide a URL, and the tool will fetch the relevant content, including text, metadata, or other embedded information, from that web page.
When Used:
Automatically called when extracting content from a provided URL, such as fetching text from articles or blog posts.
code:python-interpreter
Description:
Executes Python code directly within the platform for custom scripts, data processing, or automation. You can run Python scripts, test snippets of code, or automate tasks through coding logic.
When Used:
Automatically called when tasks involve writing or running Python scripts, such as custom data manipulations or logic-based automation.
file:text
Description:
Reads and extracts text from files, supporting only .txt and .pdf formats. To use this tool, you must provide the file IDs. (For now, only one file is supported.)
When Used:
Automatically called when text extraction from a file is requested, such as pulling content from documents or reports.
Conclusion
- Chat Completion: Best for general conversations that rely on the model’s pre-existing knowledge.
- Tool Assisted Chat Completion: Ideal for real-time, dynamic tasks and more advanced interactions, leveraging external tools to enhance functionality.
Explore APIs
To get started with Tool Assisted Chat Completion, follow this tutorial: Tool calling with Serverless Endpoints.
For more details, check out the API Reference documentations below:
Was this page helpful?