Container
Container tokenization
API Reference
Dedicated
- Overview
- POSTChat completions
- POSTCompletions
- POSTTokenization
- POSTDetokenization
- POSTImage generations
- Chat completions chunk object
- Completions chunk object
- GETList endpoints
- GETRetrieve an endpoint specification
- GETRetrieve an endpoint status
- POSTCreate an endpoint with Hugging Face model
- POSTCreate endpoint from W&B artifact
- PUTUpdate an endpoint with new configuration
- PUTTerminate an endpoint
- PUTRestart an endpoint
Container
Container
Container tokenization
By giving a text input, generate a tokenized output of token IDs.
POST
/
v1
/
tokenize
curl --request POST \
--url http://localhost:8000/v1/tokenize \
--header 'Content-Type: application/json' \
--data '{
"prompt": "What is generative AI?"
}'
{
"tokens": [
128000,
3923,
374,
1803,
1413,
15592,
30
]
}
Body
application/json
Response
200
application/json
Successfully tokenized the text.
The response is of type object
.
Was this page helpful?
curl --request POST \
--url http://localhost:8000/v1/tokenize \
--header 'Content-Type: application/json' \
--data '{
"prompt": "What is generative AI?"
}'
{
"tokens": [
128000,
3923,
374,
1803,
1413,
15592,
30
]
}