Friendli Serverless Endpoints
Tool Assisted API
Tool Assisted API enhances a model’s capabilities by integrating tools that extend its functionality beyond simple conversational interactions. By using this API, the model becomes more dynamic, providing more comprehensive and actionable responses. Currently, Friendli Serverless Endpoints supports a variety of built-in tools specifically designed for Chat Completion tasks.
What is Tool Assisted API?
Tool Assisted API enhances a model’s capabilities by integrating tools that extend its functionality beyond simple conversational interactions. By using this API, the model becomes more dynamic, providing more comprehensive and actionable responses. Currently, Friendli Serverless Endpoints supports a variety of built-in tools specifically designed for Chat Completion tasks.What is Chat Completion?
Chat completion refers to a model’s ability to generate responses in a conversation. Given a sequence of messages or conversation turns, the model processes the input and generates a response based on its internal knowledge and training data.- Example:
- User: “What is the capital of France?”
- Model: “The capital of France is Paris.”
Is Chat Completion Different from Tool Assisted Chat Completion?
Yes, Tool Assisted Chat Completion goes beyond basic chat completion by integrating external tools to enhance the conversation. This allows the model to access real-time data, perform specific tasks, and interact with external systems in ways that chat completion alone cannot achieve.- Example:
- User: “What is the weather today?”
- Model without Tool Access: Relies on pre-learned information, potentially giving outdated or generalized answers.
- Model with Tool Access: Calls a weather API to retrieve live data and responds: “The weather today in New York is 72°F with clear skies.”
- Example:
- User: “Can you extract the text from this document?” (provides a file)
- Model without Tool Access: “I cannot extract data from files directly.”
- Model with Tool Access: Extracts the text from the provided file and responds: “Using the
file:texttool, I’ve extracted the following text: [Text from the file].”
Benefits of Tool Assisted Chat Completion
Tool Assisted Chat Completion offers several advantages over basic chat completion:- Real-Time Data Access: The model can pull live information.
- Extended Capabilities: The model can perform complex tasks like running calculations, executing code, extracting text from files, and interacting with databases and APIs.
Comparison: Chat Completion vs. Tool Assisted Chat Completion
| Feature | Chat Completion | Tool Assisted Chat Completion |
|---|---|---|
| Response Type | Based on internal knowledge | Uses external tools for enhanced, real-time responses |
| Capabilities | Limited to pre-learned knowledge | Can interact with tools for data retrieval and task execution |
| Example | ”What is the weather today?” (general knowledge) | “What is the weather today?” (live API result) |
| Use Cases | General conversation and Q&A | Complex tasks like real-time updates, data analysis, file processing |
Built-In Tools
Tool Assisted API automatically selects the best tool to perform an action based on user input when a specific tool is enabled. These tools can handle various operations, such as calculations, statistical analysis, web search, file content extraction, and code execution. Below is a more detailed description of the available tools in Tool Assisted API and when they are typically used:math:calculator
Description:
Performs basic arithmetic operations like addition, subtraction, multiplication, division, and more complex calculations like and square roots or exponents.
It is useful for any tasks requiring mathematical computation.
When Used:
Automatically called when mathematical expressions or calculations are required.Whether you’re solving equations, calculating percentages, or handling financial calculations, this tool performs the task for you
web:search
Description:
Retrieves information from the web based on search queries. It fetches information based on keywords and helps gather
knowledge or insights from online sources.
When Used:
Automatically called when you ask questions or seek information that requires external research or the latest data from the web.Whether it is looking up definitions, recent news, or general web searches, this tool handles such tasks effectively.
code:python-interpreter
Description:
Executes Python code directly within the platform for custom scripts, data processing, or automation.
You can run Python scripts, test snippets of code, or automate tasks through coding logic.
When Used:
Automatically called when tasks involve writing or running Python scripts, such as custom data manipulations or logic-based automation.
Conclusion
- Chat Completion: Best for general conversations that rely on the model’s pre-existing knowledge.
- Tool Assisted Chat Completion: Ideal for real-time, dynamic tasks and more advanced interactions, leveraging external tools to enhance functionality.