LangChain Python SDK

You can use LangChain Python SDK to interact with FriendliAI. This makes migration of existing applications already using LangChain particularly easy.

How to Use

Before you start, ensure you’ve already obtained the API_KEY from the Friendli Suite > Personal Settings > API Keys. FriendliAI is fully compatible with OpenAI, so you can use the langchain-openai package by pointing it at the FriendliAI baseURL.

pip install -qU langchain-openai langchain

Instantiation

Now you can instantiate the model object and generate chat completions. Choose the example for your endpoint type:

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="zai-org/GLM-5.3",
    base_url="https://api.friendli.ai/serverless/v1",
    api_key=os.environ["API_KEY"],
)

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="YOUR_ENDPOINT_ID",
    base_url="https://api.friendli.ai/dedicated/v1",
    api_key=os.environ["API_KEY"],
)

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="YOUR_ENDPOINT_ID:YOUR_ADAPTER_ROUTE",
    base_url="https://api.friendli.ai/dedicated/v1",
    api_key=os.environ["API_KEY"],
)

Runnable Interface

FriendliAI supports both synchronous and asynchronous runnable methods to generate a response.

Synchronous Methods

result = llm.invoke("Tell me a joke.")
print(result.content)

for chunk in llm.stream("Tell me a joke."):
    print(chunk.content, end="", flush=True)

for r in llm.batch(["Tell me a joke.", "Tell me a useless fact."]):
    print(r.content, "\n\n")

Asynchronous Methods

result = await llm.ainvoke("Tell me a joke.")
print(result.content)

async for chunk in llm.astream("Tell me a joke."):
    print(chunk.content, end="", flush=True)

for r in await llm.abatch(["Tell me a joke.", "Tell me a useless fact."]):
    print(r.content, "\n\n")

Chaining

You can chain the model with a prompt template. Prompt templates convert raw user input to better input to the LLM.

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a world class technical documentation writer."),
    ("user", "{input}")
])

chain = prompt | llm

print(chain.invoke({"input": "how can langsmith help with testing?"}))

To get the string value instead of the message, add an output parser to the chain.

from langchain_core.output_parsers import StrOutputParser

output_parser = StrOutputParser()

chain = prompt | llm | output_parser

print(chain.invoke({"input": "how can langsmith help with testing?"}))

Tool Calling

Describe tools and their parameters, and let the model return a tool to invoke with the input arguments. Tool calling is extremely useful for enhancing the model’s capability to provide more comprehensive and actionable responses.

Define Tools to Use

Use the @tool decorator to define a tool. If you set parse_docstring=True, the tool will parse the docstring to extract the information of arguments.

from langchain_core.tools import tool

@tool
def add(a: int, b: int) -> int:
    """Adds a and b."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiplies a and b."""
    return a * b

tools = [add, multiply]

from langchain_core.tools import tool

@tool(parse_docstring=True)
def add(a: int, b: int) -> int:
    """Adds a and b.

    Args:
        a: The first integer.
        b: The second integer.
    """
    return a + b

@tool(parse_docstring=True)
def multiply(a: int, b: int) -> int:
    """Multiplies a and b.

    Args:
        a: The first integer.
        b: The second integer.
    """
    return a * b

tools = [add, multiply]

Bind Tools to the Model

Now models can generate a tool calling response.

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="zai-org/GLM-5.3",
    base_url="https://api.friendli.ai/serverless/v1",
    api_key=os.environ["API_KEY"],
)

llm_with_tools = llm.bind_tools(tools)

query = "What is 3 * 12? Also, what is 11 + 49?"

print(llm_with_tools.invoke(query).tool_calls)

Generate a Tool-Assisted Message

Use the tool call results to generate a message.

from langchain_core.messages import HumanMessage, ToolMessage

messages = [HumanMessage(query)]
ai_msg = llm_with_tools.invoke(messages)
messages.append(ai_msg)

for tool_call in ai_msg.tool_calls:
    selected_tool = {"add": add, "multiply": multiply}[tool_call["name"].lower()]
    tool_output = selected_tool.invoke(tool_call["args"])
    messages.append(ToolMessage(tool_output, tool_call_id=tool_call["id"]))

print(llm_with_tools.invoke(messages))

For more information on how to use tools, check out the LangChain documentation.

Models

Agents

SDKs

Tutorials

How to Use

Instantiation

Runnable Interface

Synchronous Methods

Asynchronous Methods

Chaining

Tool Calling

Define Tools to Use

Bind Tools to the Model

Generate a Tool-Assisted Message

​How to Use

​Instantiation

​Runnable Interface

​Synchronous Methods

​Asynchronous Methods

​Chaining

​Tool Calling

​Define Tools to Use

​Bind Tools to the Model

​Generate a Tool-Assisted Message

How to Use

Instantiation

Runnable Interface

Synchronous Methods

Asynchronous Methods

Chaining

Tool Calling

Define Tools to Use

Bind Tools to the Model

Generate a Tool-Assisted Message