POST
/
v1
/
tokenize
curl --request POST \
  --url http://localhost:8000/v1/tokenize \
  --header 'Content-Type: application/json' \
  --data '{
  "prompt": "What is generative AI?"
}'
{
  "tokens": [
    128000,
    3923,
    374,
    1803,
    1413,
    15592,
    30
  ]
}

Body

application/json
​
prompt
string
required

Input text prompt to tokenize.

Example:

"What is generative AI?"

​
model
string | null

Routes the request to a specific adapter.

Example:

"(adapter-route)"

Response

200
application/json
Successfully tokenized the text.
​
tokens
integer[]
required

A list of token IDs.

Was this page helpful?