Skip to main content
POST
/
serverless
/
v1
/
tokenize
Tokenization
curl --request POST \
  --url https://api.friendli.ai/serverless/v1/tokenize \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "meta-llama-3.1-8b-instruct",
  "prompt": "What is generative AI?"
}
'
{
  "tokens": [
    128000,
    3923,
    374,
    1803,
    1413,
    15592,
    30
  ]
}
By giving a text input, generate a tokenized output of token IDs. To request successfully, it is mandatory to enter a Friendli Token (e.g. flp_XXX) value in the Bearer Token field. Refer to the authentication section on our introduction page to learn how to acquire this variable and visit here to generate your token.

Authorizations

​
Authorization
string
header
required

When using Friendli Suite API for inference requests, you need to provide a Friendli Token for authentication and authorization purposes.

For more detailed information, please refer here.

Headers

​
X-Friendli-Team
string | null

ID of team to run requests as (optional parameter).

Body

application/json
​
model
string
required

Code of the model to use. See available model list.

Example:

"meta-llama-3.1-8b-instruct"

​
prompt
string
required

Input text prompt to tokenize.

Example:

"What is generative AI?"

Response

Successfully tokenized the text.

​
tokens
integer[]
required

A list of token IDs.