Dedicated Chat Completions Chunk Object

Represents a streamed chunk of a chat completions response returned by the model, based on the provided input.

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "(endpoint-id)",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": { "role": "assistant", "content": "This" },
      "finish_reason": null,
      "logprobs": null
    }
  ],
  "usage": null,
  "created": 1726294381
}

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "(endpoint-id)",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": { "content": " is" },
      "finish_reason": null,
      "logprobs": null
    }
  ],
  "usage": null,
  "created": 1726294381
}

...

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "(endpoint-id)",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": {},
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": null,
  "created": 1726294383
}

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "(endpoint-id)",
  "object": "chat.completion.chunk",
  "choices": [],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 4,
    "total_tokens": 12
  },
  "created": 1726294402
}

data: [DONE]

string

required

A unique ID of the chat completion.

object

string

required

The object type, which is always set to chat.completion.chunk.

model

string

required

The model to generate the completion. For dedicated endpoints, it returns the endpoint ID.

choices

object[]

required

Show child attributes

index

integer

required

The index of the choice in the list of generated choices.

delta

object

required

Show child attributes

role

string | null

Role of the generated message author, in this case assistant.

content

string | null

The contents of the assistant message.

tool_calls

object[] | null

Show child attributes

index

integer

required

The index of the tool call being generated.

string | null

The ID of the tool call.

type

string

required

The type of the tool, which is always set to function.

function

object

required

Show child attributes

name

string | null

The name of the function to call.

arguments

string

required

The arguments for calling the function, generated by the model in JSON format. Ensure to validate these arguments in your code before invoking the function since the model may not always produce valid JSON.

finish_reason

enum<string> | null

Termination condition of the generation. stop means the API returned the full chat completions generated by the model without running into any limits. length means the generation exceeded max_tokens or the conversation exceeded the max context length. tool_calls means the API has generated tool calls.Available options: stop, length, tool_calls

logprobs

object | null

Log probability information for the choice.

Show child attributes

content

object[] | null

A list of message content tokens with log probability information.

Show child attributes

token

string

required

The token.

logprob

number

required

The log probability of this token.

bytes

integer[] | null

A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token.

top_logprobs

object[]

required

List of the most likely tokens and their log probability, at this token position.

Show child attributes

token

string

required

The token.

logprob

number

required

The log probability of this token.

bytes

integer[] | null

usage

object | null

Show child attributes

prompt_tokens

integer

required

Number of tokens in the prompt.

completion_tokens

integer

required

Number of tokens in the generated chat completions.

total_tokens

integer

required

Total number of tokens used in the request (prompt_tokens + completion_tokens).

created

integer

required

The Unix timestamp (in seconds) for when the token is sampled.